-- Logs begin at Mon 2022-07-04 08:33:31 UTC, end at Mon 2022-07-04 09:36:26 UTC. -- Jul 04 09:00:37.750017 dv-cinder-819149-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 04 09:00:37.759071 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 4 09:00:37 2022] *** Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: nodename: dv-cinder-819149-7-iscsi Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: machine: x86_64 Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: clock source: unix Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: pcre jit disabled Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: detected number of CPU cores: 4 Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: current working directory: / Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: detected binary path: /usr/bin/uwsgi-core Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: your processes number limit is 47680 Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: your memory page size is 4096 bytes Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: detected max file descriptor number: 2048 Jul 04 09:00:37.761062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: lock engine: pthread robust mutexes Jul 04 09:00:37.761769 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: thunder lock: enabled Jul 04 09:00:37.761769 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 04 09:00:37.761769 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 04 09:00:37.804062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: Python main interpreter initialized at 0x5556dede29c0 Jul 04 09:00:37.804062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: python threads support enabled Jul 04 09:00:37.804062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: your server socket listen backlog is limited to 100 connections Jul 04 09:00:37.804062 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: your mercy for graceful operations on workers is 90 seconds Jul 04 09:00:37.804419 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: mapped 403077 bytes (393 KB) for 2 cores Jul 04 09:00:37.804419 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: *** Operational MODE: preforking *** Jul 04 09:00:37.804732 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: *** uWSGI is running in multiple interpreter mode *** Jul 04 09:00:37.804732 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: spawned uWSGI master process (pid: 80278) Jul 04 09:00:37.804945 dv-cinder-819149-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 04 09:00:37.806013 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: spawned uWSGI worker 1 (pid: 80283, cores: 1) Jul 04 09:00:37.806013 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: spawned uWSGI worker 2 (pid: 80284, cores: 1) Jul 04 09:00:37.806013 dv-cinder-819149-7-iscsi devstack@keystone.service[80278]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 04 09:00:43.177134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:00:43.177134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: from cryptography.utils import int_from_bytes Jul 04 09:00:43.177992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:00:43.177992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: from cryptography.utils import int_from_bytes Jul 04 09:00:43.178437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:00:43.178437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: from cryptography.utils import int_from_bytes Jul 04 09:00:43.178547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 04 09:00:43.178547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: from cryptography.utils import int_from_bytes Jul 04 09:00:43.225907 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 09:00:43.226073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 09:00:43.226288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] command line args: [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 09:00:43.226438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 04 09:00:43.226438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 04 09:00:43.226595 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 09:00:43.226663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] command line args: [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 04 09:00:43.226727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ================================================================================ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 09:00:43.226930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 04 09:00:43.227063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] admin_token = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.227143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ================================================================================ {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 04 09:00:43.227215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config_dir = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.227347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.227490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] admin_token = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.227567 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config_source = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.227633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] config_dir = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.227697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] debug = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.227768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.227834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.228104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] config_source = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.228104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] debug = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.228227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.228288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.228374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.228566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.228650 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.228718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.228856 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.228945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.229118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.229241 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.229308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_config_append = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.229378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.229439 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.229496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_config_append = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.229629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_dir = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.229705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.229771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.229837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_dir = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.229990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_file = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.230062 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.230264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.230386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.230470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.230541 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.230701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.230794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.230867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.230935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.231076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.231233 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.231324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.231404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.231470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.231542 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.231719 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.231860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.231970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.232082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.232223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.232325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.232425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.232499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.232601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.232683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.232869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.232999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.233094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.233224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.233383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.233600 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.233798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] publish_errors = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.233960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.234053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.234133 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] publish_errors = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.234276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.234364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.234437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.234554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.234639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.234711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.234879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.234972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.235111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] standard_threads = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.235192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.235338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.235471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] standard_threads = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.235551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.235622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.235739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.235827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.235898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_journal = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.235981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.236059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_json = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.236130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] use_journal = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.236200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_stderr = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.236269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] use_json = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.236392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_syslog = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.236477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] use_stderr = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.236558 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] use_syslog = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.236629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.236805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 04 09:00:43.236892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.237050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.237131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.237286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.237367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.237527 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.237607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.237771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.237853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.238028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.238112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.238267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.238355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.238498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.238579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.238743 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.238824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.238992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.239078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.239238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.239319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.239465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.239545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.239704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.239785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.239933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.240015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.240169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.240255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.240430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.240503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.240654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.240728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.240902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.241137 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.241228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.241384 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.241526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.241603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.241674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.241921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.242026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.242102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.242263 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.242393 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.242473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.242723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.242798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.242954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.243041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.243185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.243262 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.243414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.243498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.243649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.243734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.243892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.243968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.244132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.244216 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.244357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.244442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.244587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.244664 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.244831 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.244907 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.245062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.245141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.245295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.245372 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.245542 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.245618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.245768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.245852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.246028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.246107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.246256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.246332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.246486 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.246560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.246697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.246775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.246926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.247002 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.247155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.247239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.247387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.247463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.247618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.247694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.247845 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.247922 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.248080 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.248157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.248306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.248382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.248537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.248614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.248764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.248847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.249005 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.249107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.249240 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.249342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.249471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.249572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.249840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.250029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.250120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.250194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.250309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.250384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.250496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.250571 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.250640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.250712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.250863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.250944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.251091 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.251167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.251324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.251412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.251553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.251640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.251789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.251868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.252029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.252158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.252579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.252704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.252845 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.252922 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.253036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.253112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.253266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.253353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.253497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.253628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.253745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.253870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.254013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.254095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.254246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.254323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.254478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.254559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.254710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.254790 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.254944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.255025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.255173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.255293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.255406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.255484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.255641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.255718 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.255926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.256044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.256181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.256258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.256421 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.256500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.256659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.256745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.256896 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.256974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.257138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.257216 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.257375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.257462 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.257610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.257687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.257844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.257978 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.258110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.258270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.258348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.258418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.258564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.258642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.258797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.258882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.259021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.259111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.259253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.259333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.259481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.259558 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.259709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.259794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.259936 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.260014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.260167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.260244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.260395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.260481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.260632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.260709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.260861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.260944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.261104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.261194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.261348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.261428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.261600 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.261676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.261828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.external = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.261937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.262083 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.262161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.262324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.external = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.262401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.262557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.262662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.262853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.263029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.password = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.263109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.263301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.password = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.263413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.token = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.263532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] auth.token = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.263773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.263852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.264009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.264205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.264283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.264471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.264624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.264712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.264949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.265027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.265178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.265412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.265498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.265654 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.265814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.265911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.266158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.266238 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.266394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.266591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.266677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.266857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.266980 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.267119 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.267366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.267447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.267599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.267756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.267834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.268121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.268207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.268365 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.268498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.268635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.268859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.268938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.269132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.269265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.269370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.269610 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.269688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.269857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.270038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.270120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.270348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.270426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.270596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.270840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.270929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.271093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.271218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.271332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.271649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.271733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.271927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.272007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.272253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.272405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.272483 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.272730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.272815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.272980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.273123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.273221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.273458 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.273535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.273707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.273863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.274012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.274190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.274411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.274490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.274677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.274803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.274907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.275138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.275217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.275369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.275558 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.275638 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.275846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.275923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.276093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.276300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.276380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.276581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.276665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.276851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.277031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.277117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.277310 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.277389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.277543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.277754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.277833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.278021 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.278174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.278254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.278451 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.278529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.278710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.278796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.278968 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.279125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.279208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.279423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.279501 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.279671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.279834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.279918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.280138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.280221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.280387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.280583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.280662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.280841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.280939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.281071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.281295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.281383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.281527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.281687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.281774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.282055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.282140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.282309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.282437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.282559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.282807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.282885 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.283040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.283198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.283278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.283512 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.283590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.283767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.283976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.284063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.284228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.284339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.284465 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.284696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.284774 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.284936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.285048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.285196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.285410 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.285500 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.285661 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.285747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.285932 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.286145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.286235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.286405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.286519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.286637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.286873 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.286952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.287167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.287341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.287428 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.287698 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.287790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.287886 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.288035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.288134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.288381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.288469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.288627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.288765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.288873 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.289107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.289185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.289355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.289532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.289611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.289812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.289911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.290072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.290281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.290360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.290536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.290636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.290783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.291002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.291082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.291261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.291339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.291493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.291629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.291738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.291981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.292059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.292215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.292346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.292452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.292697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.292776 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.292938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.293087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.293165 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.293393 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.293479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.293634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.293781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.293927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.294151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.294230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.294389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.294563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.294642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.294875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.294953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.295113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.295279 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.295366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.295587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.295665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.295833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.295919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.296071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] role.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.296276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.296355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] role.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.296538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.296616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.296770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.296859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.297025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.297104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.297273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.297352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.297522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.297609 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.297767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.297963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.298096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.298271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.298349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.298519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.298669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.298747 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.298990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.299068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.299237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.299378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.299483 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.299727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.299805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.300116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.300194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.300352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.300537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.300621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.300838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.300916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.301086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.301175 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.301335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.301414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.301570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.301649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.301823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.301951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.302099 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.302311 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.302438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.302558 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.302675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.302829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.303064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.303143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.303426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.303505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.303701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.303802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.303950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.304189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.304268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.304423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.304556 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.304656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.304892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.304970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.305131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.305354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.305442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.305604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.305681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.305862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.305998 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.306131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.306223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.306370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.306450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.306614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.306778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.306856 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.307089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.307168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.307325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.307495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.307573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.307781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.307867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.308027 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.308208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.308292 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.308490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.308588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.308733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.308961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.309039 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.309289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.connection = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.309376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.309538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.309767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.309854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.310037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.310118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.310281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.310360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.310518 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.310605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.310758 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.310837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.310997 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.311177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.311255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.311477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.311555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.311718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.311902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.311981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.312187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.312273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.312436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.312621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.312699 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.312901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.312999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.313132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.313372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.313451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.313605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.313769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.313848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 09:00:43.314222 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.314604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.314984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.315634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.315912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.316162 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.316411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.316658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.316903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.317318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.317554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.317803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.318078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.318326 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.318630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.318865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.319124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.319372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.319610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.319842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.320073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.320302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.320550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.320815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.321063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.321302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.321538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.321768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.322026 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.322264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.322512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.322747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.323001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.323247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.323484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.323725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.323965 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.324208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.324441 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.324678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.324909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.325152 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.325399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.325628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.325866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.326188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.326446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.326688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.326932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.327176 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.327414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.327647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.327880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.328112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.328346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.328587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.328819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.329065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.329297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.329540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.329772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.330041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.330270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.330508 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.330741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 04 09:00:43.330908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 04 09:00:43.345206 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.345439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.345672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.354199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.356852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.357096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.357332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.365913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.382755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.382996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.384378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.384610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.525514 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.525706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.574545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.574751 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.576097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.576440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.639312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.639589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.639689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.639756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.639841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.639919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.639982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.640054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.640116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.640198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.640406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.640531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.665074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.665231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.665325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.665415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.665480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.665717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.665843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.665958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.666052 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.666121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.666216 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.666346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.666438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.666597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.677428 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.677680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.690129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.690351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.690606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.690729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.690806 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.690870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.691034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.691172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.691244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.691319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.691427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80283) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.691516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.691710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.691926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 04 09:00:43.708649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.708864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:00:43.709317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.709449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.709551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:00:43.709620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:00:43.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.709924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.710017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.710120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 04 09:00:43.710222 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.710410 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.710540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.710638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.710924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.711151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.721189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.721461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.721726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.721852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.722031 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.722183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.722362 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.722640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.730420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.730923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.735449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.735880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.736400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.736864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.737364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.737804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.738312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.738983 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.739270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.739763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.740300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.741282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.741581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.741869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.742195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.742498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.742762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.743039 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.743313 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.743585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.743924 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.744252 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.771418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.771709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.773209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.773335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.773461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.773552 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.778765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.778886 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.778956 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.779099 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.779355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.779480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.786588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.786713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.786845 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.787017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.791327 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.791554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.791690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.791841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.791989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.792092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.792182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.792389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.792514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.792671 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.792812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.792885 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.792963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.793179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.793304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.793389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.811772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.812050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.812178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.812302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.816573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.817097 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.820338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.820628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.820884 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.821179 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.825704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.826328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.829312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.829928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.832927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.833189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.833503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.833629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.833772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.833868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.834224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.834350 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.834478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.834560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.834801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.835079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.853441 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.853722 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.853994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.854094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.854290 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.854416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.854580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.854847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.862080 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.862691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.863760 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.864028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.864413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.864687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.867273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.867551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.867825 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.868110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.868235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.868352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.868568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.868801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.869083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.869358 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.881990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.882265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.882550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.882822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.882946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.883100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.883239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.883455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.883736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.884014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.884718 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.885005 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.885644 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.885956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.902788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.904034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.906038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.907337 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.909296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.910651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.911038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.912523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.912649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.912954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.913248 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.914378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.914666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.914949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.920498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.922208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.922333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.923843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.925369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.925652 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.926069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.926342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.927078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.927364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.927755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.928062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.935360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.935651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.935915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.937278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.937567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.937847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.943737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.944110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.944391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.944662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.944931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.945308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.945585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.945835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.945984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80283) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 04 09:00:43.946183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.946326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.946409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.946485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.946695 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.946820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.946933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.947033 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.947193 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.947332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.947465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.947607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.947881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.948162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.948432 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.948716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.948976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.949242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.949509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80283) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 04 09:00:43.984782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5556dede29c0 pid: 80284 (default app) Jul 04 09:00:43.987572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5556dede29c0 pid: 80283 (default app) Jul 04 09:00:43.988179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914ada6-b443-46c8-89fb-0674252db94f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:43.988472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914ada6-b443-46c8-89fb-0674252db94f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:43.988761 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914ada6-b443-46c8-89fb-0674252db94f None None] PATH_INFO: `/v3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:43.993228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Mon Jul 4 09:00:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:00:51.624882 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c2824-6984-4c35-942b-7ce35cca7266 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:51.625811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c2824-6984-4c35-942b-7ce35cca7266 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:51.625811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c2824-6984-4c35-942b-7ce35cca7266 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:51.633098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:00:51 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:00:51.641568 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6593-8b1f-47fa-9c5e-0930723f2186 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:51.642034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6593-8b1f-47fa-9c5e-0930723f2186 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:51.642174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6593-8b1f-47fa-9c5e-0930723f2186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:51.717625 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: INFO dbcounter [None req-a33f6593-8b1f-47fa-9c5e-0930723f2186 None None] Registered counter for database keystone Jul 04 09:00:51.742063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG oslo_db.sqlalchemy.engines [None req-a33f6593-8b1f-47fa-9c5e-0930723f2186 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80284) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 09:00:51.754495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writer thread running {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 09:00:51.956488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG passlib.handlers.bcrypt [None req-a33f6593-8b1f-47fa-9c5e-0930723f2186 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80284) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 09:00:51.956979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG passlib.handlers.bcrypt [None req-a33f6593-8b1f-47fa-9c5e-0930723f2186 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80284) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 09:00:52.014282 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a33f6593-8b1f-47fa-9c5e-0930723f2186 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:00:52.015942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a33f6593-8b1f-47fa-9c5e-0930723f2186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:00:52.153694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:00:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 514 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 04 09:00:52.345435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c2824-6984-4c35-942b-7ce35cca7266 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:52.419219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: INFO dbcounter [None req-139c2824-6984-4c35-942b-7ce35cca7266 None None] Registered counter for database keystone Jul 04 09:00:52.440440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG oslo_db.sqlalchemy.engines [None req-139c2824-6984-4c35-942b-7ce35cca7266 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80283) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 04 09:00:52.451547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writer thread running {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 04 09:00:52.579873 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c2824-6984-4c35-942b-7ce35cca7266 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:52.585465 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6a0b6-00fe-42dd-a34d-e07ad86a8698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:52.586173 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6a0b6-00fe-42dd-a34d-e07ad86a8698 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:52.586460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6a0b6-00fe-42dd-a34d-e07ad86a8698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:52.586731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6a0b6-00fe-42dd-a34d-e07ad86a8698 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:52.596134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6a0b6-00fe-42dd-a34d-e07ad86a8698 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:00:52.648023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: WARNING keystone.common.rbac_enforcer.enforcer [None req-aad6a0b6-00fe-42dd-a34d-e07ad86a8698 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 09:00:52.648606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6a0b6-00fe-42dd-a34d-e07ad86a8698 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:00:52.658761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Jul 4 09:00:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:00:52.666261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f6593-8b1f-47fa-9c5e-0930723f2186 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:52.685728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f6593-8b1f-47fa-9c5e-0930723f2186 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:52.691711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29df9f5-01bc-4c13-90b6-a5b4bd8c6c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:52.692847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29df9f5-01bc-4c13-90b6-a5b4bd8c6c63 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:52.693181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29df9f5-01bc-4c13-90b6-a5b4bd8c6c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:52.693463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29df9f5-01bc-4c13-90b6-a5b4bd8c6c63 admin admin] PATH_INFO: `/v3/projects/3f0a53c420a84618ab851430c484608a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:52.697830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29df9f5-01bc-4c13-90b6-a5b4bd8c6c63 admin admin] RBAC: Authorizing `identity:get_project(project_id=3f0a53c420a84618ab851430c484608a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:00:52.749041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e29df9f5-01bc-4c13-90b6-a5b4bd8c6c63 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 04 09:00:52.749383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29df9f5-01bc-4c13-90b6-a5b4bd8c6c63 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:00:52.754454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Mon Jul 4 09:00:52 2022] GET /identity/v3/projects/3f0a53c420a84618ab851430c484608a => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:00:54.743001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e02cec-e8be-4bd5-8206-c4484cae5df4 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:54.743578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e02cec-e8be-4bd5-8206-c4484cae5df4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:54.743578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e02cec-e8be-4bd5-8206-c4484cae5df4 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:54.747756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:00:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:00:54.754870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c4650-fc0e-4db6-9a95-73b584afdebe None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:54.755743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c4650-fc0e-4db6-9a95-73b584afdebe None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:54.755862 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c4650-fc0e-4db6-9a95-73b584afdebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:54.777205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ec7c4650-fc0e-4db6-9a95-73b584afdebe None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:00:54.778807 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ec7c4650-fc0e-4db6-9a95-73b584afdebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:00:54.790920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:00:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 04 09:00:54.817774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e02cec-e8be-4bd5-8206-c4484cae5df4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:54.832227 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e02cec-e8be-4bd5-8206-c4484cae5df4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:54.837493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4d1ad-2294-41a1-80d0-4a460dc4d2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:54.838155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4d1ad-2294-41a1-80d0-4a460dc4d2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:54.838468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4d1ad-2294-41a1-80d0-4a460dc4d2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:54.838750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4d1ad-2294-41a1-80d0-4a460dc4d2c2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:54.846913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4d1ad-2294-41a1-80d0-4a460dc4d2c2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:00:54.848720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4d1ad-2294-41a1-80d0-4a460dc4d2c2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:00:54.857915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Jul 4 09:00:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:00:54.864456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c4650-fc0e-4db6-9a95-73b584afdebe None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:54.879713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c4650-fc0e-4db6-9a95-73b584afdebe None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:54.888797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d9a55-5413-4833-9b87-32fb141b03b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:54.890230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d9a55-5413-4833-9b87-32fb141b03b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:54.890690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d9a55-5413-4833-9b87-32fb141b03b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:54.891166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d9a55-5413-4833-9b87-32fb141b03b2 admin admin] PATH_INFO: `/v3/users/78c2f3799fc8459a9a6e05f14a552498` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:54.893595 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d9a55-5413-4833-9b87-32fb141b03b2 admin admin] RBAC: Authorizing `identity:get_user(user_id=78c2f3799fc8459a9a6e05f14a552498)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:00:54.896654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d9a55-5413-4833-9b87-32fb141b03b2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:00:54.898268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Mon Jul 4 09:00:54 2022] GET /identity/v3/users/78c2f3799fc8459a9a6e05f14a552498 => generated 247 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:00:56.952816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b8484-4ebe-4ed8-b6da-4f262ce2d23e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:56.953394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b8484-4ebe-4ed8-b6da-4f262ce2d23e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:56.953394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b8484-4ebe-4ed8-b6da-4f262ce2d23e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:56.954883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:00:56.960061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdce4d-590e-446e-a804-eacf765eb901 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:56.960496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdce4d-590e-446e-a804-eacf765eb901 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:56.960703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdce4d-590e-446e-a804-eacf765eb901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:56.978688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-06bdce4d-590e-446e-a804-eacf765eb901 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:00:56.979861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-06bdce4d-590e-446e-a804-eacf765eb901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:00:56.986983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75907a-4aff-45d6-bedf-727419d86b8c None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:56.987276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75907a-4aff-45d6-bedf-727419d86b8c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:56.987996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75907a-4aff-45d6-bedf-727419d86b8c None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:56.989355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:00:56.994100 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa3eb8-4830-4b83-908c-b6183897b966 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:56.994393 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa3eb8-4830-4b83-908c-b6183897b966 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:56.994673 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa3eb8-4830-4b83-908c-b6183897b966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:57.035713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG passlib.handlers.bcrypt [None req-76fa3eb8-4830-4b83-908c-b6183897b966 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80283) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 04 09:00:57.036128 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG passlib.handlers.bcrypt [None req-76fa3eb8-4830-4b83-908c-b6183897b966 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80283) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 04 09:00:57.062938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:00:57.082131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-76fa3eb8-4830-4b83-908c-b6183897b966 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:00:57.083913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-76fa3eb8-4830-4b83-908c-b6183897b966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:00:57.086139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdce4d-590e-446e-a804-eacf765eb901 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:57.110015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:00:57.188424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:00:57.195145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdce4d-590e-446e-a804-eacf765eb901 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:57.211020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa3eb8-4830-4b83-908c-b6183897b966 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:57.228464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f57b6-04fc-4d7a-adb6-b03c8dfc4c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:57.229280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f57b6-04fc-4d7a-adb6-b03c8dfc4c1a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:57.229566 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f57b6-04fc-4d7a-adb6-b03c8dfc4c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:57.229856 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f57b6-04fc-4d7a-adb6-b03c8dfc4c1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:57.240941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f57b6-04fc-4d7a-adb6-b03c8dfc4c1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-7d2f57b6-04fc-4d7a-adb6-b03c8dfc4c1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:00:57.252432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:00:57.253412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:00:57.254300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:00:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:00:57.259494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f57b6-04fc-4d7a-adb6-b03c8dfc4c1a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:57.317197 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa3eb8-4830-4b83-908c-b6183897b966 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:57.319555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:00:57.341343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:00:57.355162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e5b3a-5c10-4d6f-9ea5-a22eef82b2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:57.356076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e5b3a-5c10-4d6f-9ea5-a22eef82b2dd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:57.356366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e5b3a-5c10-4d6f-9ea5-a22eef82b2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:57.356640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e5b3a-5c10-4d6f-9ea5-a22eef82b2dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:57.361060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f57b6-04fc-4d7a-adb6-b03c8dfc4c1a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:57.395765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623463a2-2f0b-45f8-a7bb-e023491651c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:57.396675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623463a2-2f0b-45f8-a7bb-e023491651c6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:57.397023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623463a2-2f0b-45f8-a7bb-e023491651c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:57.397302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623463a2-2f0b-45f8-a7bb-e023491651c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:57.398106 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623463a2-2f0b-45f8-a7bb-e023491651c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:00:57.398415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623463a2-2f0b-45f8-a7bb-e023491651c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:00:57.399824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623463a2-2f0b-45f8-a7bb-e023491651c6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:00:57.411597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:00:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:00:57.417394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623463a2-2f0b-45f8-a7bb-e023491651c6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:57.423338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e5b3a-5c10-4d6f-9ea5-a22eef82b2dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:00:57.424878 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e5b3a-5c10-4d6f-9ea5-a22eef82b2dd None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:00:57.515800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623463a2-2f0b-45f8-a7bb-e023491651c6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:57.524162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:00:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:00:57.529980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e5b3a-5c10-4d6f-9ea5-a22eef82b2dd None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:57.532708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:00:57.548899 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8049ca-94d0-4a8d-a665-60e2937402c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:57.549790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8049ca-94d0-4a8d-a665-60e2937402c2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:57.550110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8049ca-94d0-4a8d-a665-60e2937402c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:57.550393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8049ca-94d0-4a8d-a665-60e2937402c2 None admin] PATH_INFO: `/v3/users/78c2f3799fc8459a9a6e05f14a552498` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:57.551635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8049ca-94d0-4a8d-a665-60e2937402c2 None admin] RBAC: Authorizing `identity:get_user(user_id=78c2f3799fc8459a9a6e05f14a552498)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:00:57.553714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8049ca-94d0-4a8d-a665-60e2937402c2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:00:57.555432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:00:57 2022] GET /identity/v3/users/78c2f3799fc8459a9a6e05f14a552498 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:00:57.562778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8049ca-94d0-4a8d-a665-60e2937402c2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:57.578699 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:00:57.634213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e5b3a-5c10-4d6f-9ea5-a22eef82b2dd None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:57.666281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8049ca-94d0-4a8d-a665-60e2937402c2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:57.668376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f4192-c00e-4940-b384-009c736362f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:57.669209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4192-c00e-4940-b384-009c736362f8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:57.669497 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4192-c00e-4940-b384-009c736362f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:57.669775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4192-c00e-4940-b384-009c736362f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:57.688887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f4192-c00e-4940-b384-009c736362f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:00:57.690516 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f4192-c00e-4940-b384-009c736362f8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:00:57.703090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f04669-34a0-4d1d-8a34-c2354c2d9538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:57.703893 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f04669-34a0-4d1d-8a34-c2354c2d9538 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:57.704181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f04669-34a0-4d1d-8a34-c2354c2d9538 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:57.704458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f04669-34a0-4d1d-8a34-c2354c2d9538 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:57.705660 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f04669-34a0-4d1d-8a34-c2354c2d9538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:00:57.707143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f04669-34a0-4d1d-8a34-c2354c2d9538 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:00:57.708806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:00:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-b10f4192-c00e-4940-b384-009c736362f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:00:57.711413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:00:57.712266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:00:57.713123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:00:57.713123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:00:57.713123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:00:57.713123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:00:57.713123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 9/19] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:00:57.718297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f04669-34a0-4d1d-8a34-c2354c2d9538 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:57.719017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f4192-c00e-4940-b384-009c736362f8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:00:57.777761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:00:57.795314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:00:57.815046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f04669-34a0-4d1d-8a34-c2354c2d9538 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:57.821355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f4192-c00e-4940-b384-009c736362f8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:00:57.848198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cc6d8-07db-46a9-82e9-52f8feb395ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:57.848851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cc6d8-07db-46a9-82e9-52f8feb395ff None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:57.849190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cc6d8-07db-46a9-82e9-52f8feb395ff None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:57.849532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cc6d8-07db-46a9-82e9-52f8feb395ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:57.850339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cc6d8-07db-46a9-82e9-52f8feb395ff None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.domain.id=default, user.id=78c2f3799fc8459a9a6e05f14a552498) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:00:57.850865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cc6d8-07db-46a9-82e9-52f8feb395ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:00:57.852089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cc6d8-07db-46a9-82e9-52f8feb395ff None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:00:57.855375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff37fc-0cd6-4d55-8d49-40e543654096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:00:57.856059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff37fc-0cd6-4d55-8d49-40e543654096 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:00:57.856346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff37fc-0cd6-4d55-8d49-40e543654096 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:00:57.856628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff37fc-0cd6-4d55-8d49-40e543654096 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:00:57.857361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff37fc-0cd6-4d55-8d49-40e543654096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:00:57.857679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff37fc-0cd6-4d55-8d49-40e543654096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:00:57.859119 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff37fc-0cd6-4d55-8d49-40e543654096 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:00:57.860086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Mon Jul 4 09:00:57 2022] GET /identity/v3/role_assignments?user.id=78c2f3799fc8459a9a6e05f14a552498&scope.domain.id=default&role.id=2302517080ac42578aa81c606dd0ff2f => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:00:57.879212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:00.167387 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c4221-3604-4071-93cd-e3946a2783c9 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:00.167889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c4221-3604-4071-93cd-e3946a2783c9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:00.167995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c4221-3604-4071-93cd-e3946a2783c9 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:00.169307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:00.174858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d876340-f261-4521-a64a-763180750619 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:00.175283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d876340-f261-4521-a64a-763180750619 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:00.175466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d876340-f261-4521-a64a-763180750619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:00.203809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7d876340-f261-4521-a64a-763180750619 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:00.205042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7d876340-f261-4521-a64a-763180750619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:00.297726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 11/23] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:00.332913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c4221-3604-4071-93cd-e3946a2783c9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:00.479200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c4221-3604-4071-93cd-e3946a2783c9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:00.518467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764ab7d-328d-48c7-9e43-ea3c96cd0b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:00.519271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764ab7d-328d-48c7-9e43-ea3c96cd0b3b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:00.519557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764ab7d-328d-48c7-9e43-ea3c96cd0b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:00.519834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764ab7d-328d-48c7-9e43-ea3c96cd0b3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:00.530175 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764ab7d-328d-48c7-9e43-ea3c96cd0b3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a764ab7d-328d-48c7-9e43-ea3c96cd0b3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:00.538069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:00.538912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:00.539687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 13/24] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:01:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:00.546668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d876340-f261-4521-a64a-763180750619 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:00.659082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:00.662594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d1098-6c50-4d8b-bac1-0c6e2e09f326 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:00.662904 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d1098-6c50-4d8b-bac1-0c6e2e09f326 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:00.663174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d1098-6c50-4d8b-bac1-0c6e2e09f326 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:00.666630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 14/25] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:00.670831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c441b7-eba1-4d4e-b852-8524f06bc804 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:00.671125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c441b7-eba1-4d4e-b852-8524f06bc804 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:00.671413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c441b7-eba1-4d4e-b852-8524f06bc804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:00.691066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:00.710942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c2c441b7-eba1-4d4e-b852-8524f06bc804 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:00.712180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c2c441b7-eba1-4d4e-b852-8524f06bc804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:00.754108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d876340-f261-4521-a64a-763180750619 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:00.795028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3de424-de16-4616-b6d3-46d41ed99c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:00.795716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3de424-de16-4616-b6d3-46d41ed99c0b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:00.795996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3de424-de16-4616-b6d3-46d41ed99c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:00.796271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3de424-de16-4616-b6d3-46d41ed99c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:00.797031 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3de424-de16-4616-b6d3-46d41ed99c0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:00.797336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3de424-de16-4616-b6d3-46d41ed99c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:00.798864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3de424-de16-4616-b6d3-46d41ed99c0b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:00.813183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 12/26] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:00.814531 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fa30e-d4df-46a3-b4c1-4c1529b98519 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:00.814819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fa30e-d4df-46a3-b4c1-4c1529b98519 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:00.815096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fa30e-d4df-46a3-b4c1-4c1529b98519 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:00.816562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 13/27] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:00.817774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061fd69-9325-44de-a138-f8b3fd787a88 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:00.818084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061fd69-9325-44de-a138-f8b3fd787a88 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:00.818403 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061fd69-9325-44de-a138-f8b3fd787a88 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:00.819791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 14/28] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:00.824692 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061fd69-9325-44de-a138-f8b3fd787a88 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:00.843345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 15/29] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:00.844998 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3284cf-ecf4-484e-89ae-9148cc1b2395 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:00.845300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3284cf-ecf4-484e-89ae-9148cc1b2395 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:00.845586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3284cf-ecf4-484e-89ae-9148cc1b2395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:00.862694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-db3284cf-ecf4-484e-89ae-9148cc1b2395 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:00.863891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-db3284cf-ecf4-484e-89ae-9148cc1b2395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:00.938143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061fd69-9325-44de-a138-f8b3fd787a88 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:00.939636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 16/30] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:00.941003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5765bc-0b3e-420d-b621-f21834881c92 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:00.946542 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5765bc-0b3e-420d-b621-f21834881c92 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:00.946843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5765bc-0b3e-420d-b621-f21834881c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:00.949569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:00.978951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-af5765bc-0b3e-420d-b621-f21834881c92 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:00.979137 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f7ec4-95f2-4233-9a07-95259c44a893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:00.980081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f7ec4-95f2-4233-9a07-95259c44a893 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:00.980192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-af5765bc-0b3e-420d-b621-f21834881c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:00.980366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f7ec4-95f2-4233-9a07-95259c44a893 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:00.980645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f7ec4-95f2-4233-9a07-95259c44a893 None admin] PATH_INFO: `/v3/users/78c2f3799fc8459a9a6e05f14a552498` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:00.982626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f7ec4-95f2-4233-9a07-95259c44a893 None admin] RBAC: Authorizing `identity:get_user(user_id=78c2f3799fc8459a9a6e05f14a552498)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:00.985215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f7ec4-95f2-4233-9a07-95259c44a893 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:00.987376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 15/31] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:00 2022] GET /identity/v3/users/78c2f3799fc8459a9a6e05f14a552498 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:00.988152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f7ec4-95f2-4233-9a07-95259c44a893 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:01.046677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:01.056980 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 17/32] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:01.058055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5765bc-0b3e-420d-b621-f21834881c92 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:01.143931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f7ec4-95f2-4233-9a07-95259c44a893 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:01.179978 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb3a0-61df-4412-ac18-9d086422da4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:01.181554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb3a0-61df-4412-ac18-9d086422da4d None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:01.182412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb3a0-61df-4412-ac18-9d086422da4d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:01.182912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb3a0-61df-4412-ac18-9d086422da4d None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:01.183917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcb3a0-61df-4412-ac18-9d086422da4d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:01.185809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcb3a0-61df-4412-ac18-9d086422da4d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:01.188417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5765bc-0b3e-420d-b621-f21834881c92 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:01.225203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d03b7-9490-4764-a05a-1d22c8c566d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:01.226066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d03b7-9490-4764-a05a-1d22c8c566d0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:01.226358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d03b7-9490-4764-a05a-1d22c8c566d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:01.226639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d03b7-9490-4764-a05a-1d22c8c566d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:01.237472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d03b7-9490-4764-a05a-1d22c8c566d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:01.238996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d03b7-9490-4764-a05a-1d22c8c566d0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-794d03b7-9490-4764-a05a-1d22c8c566d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:01:01.258494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:01.259366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:01.260190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:01:01.260190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:01:01.260190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:01:01.260190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:01.260190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 18/33] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:01.260944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d03b7-9490-4764-a05a-1d22c8c566d0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:01.309102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:01.324736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 16/34] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 4 09:01:00 2022] POST /identity/v3/roles => generated 225 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:01.325920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb3a0-61df-4412-ac18-9d086422da4d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:01.365288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:01.371035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d03b7-9490-4764-a05a-1d22c8c566d0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:01.409156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61963635-f97f-4f81-8131-8a6e2546a536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:01.410348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61963635-f97f-4f81-8131-8a6e2546a536 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:01.410781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61963635-f97f-4f81-8131-8a6e2546a536 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:01.411217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61963635-f97f-4f81-8131-8a6e2546a536 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:01.412937 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61963635-f97f-4f81-8131-8a6e2546a536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:01.414873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61963635-f97f-4f81-8131-8a6e2546a536 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:01.416827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 19/35] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:01.418101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61963635-f97f-4f81-8131-8a6e2546a536 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:01.439326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb3a0-61df-4412-ac18-9d086422da4d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:01.479305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b4216-938b-492f-aafe-d8b6b133946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:01.480184 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b4216-938b-492f-aafe-d8b6b133946e None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:01.480565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b4216-938b-492f-aafe-d8b6b133946e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:01.480855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b4216-938b-492f-aafe-d8b6b133946e None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:01.481752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b4216-938b-492f-aafe-d8b6b133946e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:01.484230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b4216-938b-492f-aafe-d8b6b133946e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:01.511317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 17/36] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 4 09:01:01 2022] POST /identity/v3/roles => generated 219 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:01.511317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d68c9-b6db-46bf-82db-4d4bd5017859 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:01.511317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d68c9-b6db-46bf-82db-4d4bd5017859 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:01.511317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d68c9-b6db-46bf-82db-4d4bd5017859 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:01.511317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 18/37] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:01.512010 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d68c9-b6db-46bf-82db-4d4bd5017859 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:01.543690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61963635-f97f-4f81-8131-8a6e2546a536 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:01.567812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:01.605843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c20d95-4061-4b8b-a21d-5e09a3810782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:01.606595 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20d95-4061-4b8b-a21d-5e09a3810782 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:01.606885 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20d95-4061-4b8b-a21d-5e09a3810782 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:01.607165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20d95-4061-4b8b-a21d-5e09a3810782 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:01.607928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c20d95-4061-4b8b-a21d-5e09a3810782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:01.608241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c20d95-4061-4b8b-a21d-5e09a3810782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:01.609767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c20d95-4061-4b8b-a21d-5e09a3810782 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:01.630614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 20/38] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:01.632193 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358da74d-02e7-4e70-b8a6-c76efa981aee None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:01.632489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358da74d-02e7-4e70-b8a6-c76efa981aee None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:01.632772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358da74d-02e7-4e70-b8a6-c76efa981aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:01.665540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-358da74d-02e7-4e70-b8a6-c76efa981aee None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:01.666901 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-358da74d-02e7-4e70-b8a6-c76efa981aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:01.677181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:01.686973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d68c9-b6db-46bf-82db-4d4bd5017859 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:01.725793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f597eb-6b7e-42e8-b643-3cfbe47f3d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:01.726728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f597eb-6b7e-42e8-b643-3cfbe47f3d33 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:01.727040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f597eb-6b7e-42e8-b643-3cfbe47f3d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:01.727295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f597eb-6b7e-42e8-b643-3cfbe47f3d33 None admin] PATH_INFO: `/v3/domains/default/users/78c2f3799fc8459a9a6e05f14a552498/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:01.729616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f597eb-6b7e-42e8-b643-3cfbe47f3d33 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=78c2f3799fc8459a9a6e05f14a552498, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:01.731149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f597eb-6b7e-42e8-b643-3cfbe47f3d33 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:01.753099 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 19/39] 10.222.0.26 () {58 vars in 1361 bytes} [Mon Jul 4 09:01:01 2022] PUT /identity/v3/domains/default/users/78c2f3799fc8459a9a6e05f14a552498/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:01.753404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 21/40] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:01.754199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f597eb-6b7e-42e8-b643-3cfbe47f3d33 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:01.777176 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358da74d-02e7-4e70-b8a6-c76efa981aee None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:01.881686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f597eb-6b7e-42e8-b643-3cfbe47f3d33 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:01.895059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358da74d-02e7-4e70-b8a6-c76efa981aee None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:01.898249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:01.926937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252cc35-d9eb-4f27-a084-9ceb6a45cd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:01.928304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252cc35-d9eb-4f27-a084-9ceb6a45cd1d None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:01.928597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252cc35-d9eb-4f27-a084-9ceb6a45cd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:01.928879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252cc35-d9eb-4f27-a084-9ceb6a45cd1d None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:01.929626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252cc35-d9eb-4f27-a084-9ceb6a45cd1d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:01.931844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252cc35-d9eb-4f27-a084-9ceb6a45cd1d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:01.934754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fecedf-ab73-4019-aa3c-99c269fcbe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:01.935436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fecedf-ab73-4019-aa3c-99c269fcbe29 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:01.935726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fecedf-ab73-4019-aa3c-99c269fcbe29 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:01.935999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fecedf-ab73-4019-aa3c-99c269fcbe29 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:01.936784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fecedf-ab73-4019-aa3c-99c269fcbe29 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:01.938200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fecedf-ab73-4019-aa3c-99c269fcbe29 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:01.967304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 20/41] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:01:01 2022] POST /identity/v3/projects => generated 301 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:02.097169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f6dbd-21d9-4981-aadb-91af75471c41 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:02.097580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f6dbd-21d9-4981-aadb-91af75471c41 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:02.097776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f6dbd-21d9-4981-aadb-91af75471c41 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:02.099730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 21/42] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:02.106016 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0171c25-cf09-41be-a39c-e7555c266049 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:02.106324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0171c25-cf09-41be-a39c-e7555c266049 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:02.106601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0171c25-cf09-41be-a39c-e7555c266049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:02.125585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d0171c25-cf09-41be-a39c-e7555c266049 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:02.129241 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d0171c25-cf09-41be-a39c-e7555c266049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:02.138221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:02.173628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 22/43] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 4 09:01:01 2022] POST /identity/v3/roles => generated 223 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:02.212969 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 22/44] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:02.237138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fecedf-ab73-4019-aa3c-99c269fcbe29 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:02.354851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fecedf-ab73-4019-aa3c-99c269fcbe29 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:02.392066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c584e-a011-4470-8c51-713ca1f7732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:02.392814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c584e-a011-4470-8c51-713ca1f7732b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:02.393110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c584e-a011-4470-8c51-713ca1f7732b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:02.393395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c584e-a011-4470-8c51-713ca1f7732b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:02.394744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c584e-a011-4470-8c51-713ca1f7732b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:02.396653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c584e-a011-4470-8c51-713ca1f7732b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:02.398490 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 23/45] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:02.403774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0171c25-cf09-41be-a39c-e7555c266049 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:02.507915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0171c25-cf09-41be-a39c-e7555c266049 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:02.545079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b5a13-50de-464b-afd1-a9a2196759f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:02.545753 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b5a13-50de-464b-afd1-a9a2196759f8 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:02.546141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b5a13-50de-464b-afd1-a9a2196759f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:02.546425 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b5a13-50de-464b-afd1-a9a2196759f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:02.547155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b5a13-50de-464b-afd1-a9a2196759f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:02.548670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b5a13-50de-464b-afd1-a9a2196759f8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:02.557642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:02.576281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 23/46] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:01:02 2022] POST /identity/v3/projects => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:03.838487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5c3ed-126c-42ff-aec7-2afdf04b835f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:03.839041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5c3ed-126c-42ff-aec7-2afdf04b835f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:03.839041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5c3ed-126c-42ff-aec7-2afdf04b835f None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:03.840445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 24/47] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:03.845906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90f9c5-7dc1-43e3-88e6-456de36c5af6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:03.846375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90f9c5-7dc1-43e3-88e6-456de36c5af6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:03.846466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90f9c5-7dc1-43e3-88e6-456de36c5af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:03.864135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0c90f9c5-7dc1-43e3-88e6-456de36c5af6 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:03.865568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0c90f9c5-7dc1-43e3-88e6-456de36c5af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:03.937526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 24/48] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:03.960442 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5c3ed-126c-42ff-aec7-2afdf04b835f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:03.992623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:04.069127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5c3ed-126c-42ff-aec7-2afdf04b835f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:04.104782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08be7-c526-4ed0-8f93-6610928cd720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:04.105511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08be7-c526-4ed0-8f93-6610928cd720 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:04.105799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08be7-c526-4ed0-8f93-6610928cd720 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:04.106114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08be7-c526-4ed0-8f93-6610928cd720 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:04.116864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c08be7-c526-4ed0-8f93-6610928cd720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c4c08be7-c526-4ed0-8f93-6610928cd720 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:04.124529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:04.125417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:04.131721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 25/49] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:01:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:04.132364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90f9c5-7dc1-43e3-88e6-456de36c5af6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:04.343494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90f9c5-7dc1-43e3-88e6-456de36c5af6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:04.364047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:04.380629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff24e2c-96b6-4bd5-b99a-0ebfb26bfa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:04.381306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff24e2c-96b6-4bd5-b99a-0ebfb26bfa88 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:04.381591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff24e2c-96b6-4bd5-b99a-0ebfb26bfa88 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:04.381868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff24e2c-96b6-4bd5-b99a-0ebfb26bfa88 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:04.382654 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff24e2c-96b6-4bd5-b99a-0ebfb26bfa88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:04.382961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff24e2c-96b6-4bd5-b99a-0ebfb26bfa88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:04.384342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff24e2c-96b6-4bd5-b99a-0ebfb26bfa88 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:04.392724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 25/50] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:04.398739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08be7-c526-4ed0-8f93-6610928cd720 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:04.495447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:04.506734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08be7-c526-4ed0-8f93-6610928cd720 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:04.542911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8602b1-f584-44fa-b2c1-f8b710883602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:04.543802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8602b1-f584-44fa-b2c1-f8b710883602 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:04.544095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8602b1-f584-44fa-b2c1-f8b710883602 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:04.544366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8602b1-f584-44fa-b2c1-f8b710883602 None admin] PATH_INFO: `/v3/users/78c2f3799fc8459a9a6e05f14a552498` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:04.545666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8602b1-f584-44fa-b2c1-f8b710883602 None admin] RBAC: Authorizing `identity:get_user(user_id=78c2f3799fc8459a9a6e05f14a552498)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:04.547197 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8602b1-f584-44fa-b2c1-f8b710883602 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:04.548855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 26/51] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:04 2022] GET /identity/v3/users/78c2f3799fc8459a9a6e05f14a552498 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:04.554721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff24e2c-96b6-4bd5-b99a-0ebfb26bfa88 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:04.559734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af3bb6-0b1a-48b4-abb9-9b1086b3c437 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:04.560023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af3bb6-0b1a-48b4-abb9-9b1086b3c437 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:04.560305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af3bb6-0b1a-48b4-abb9-9b1086b3c437 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:04.561674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 27/52] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:04.566738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee60ccc-eb4b-4d13-9950-adefa14c7152 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:04.567021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee60ccc-eb4b-4d13-9950-adefa14c7152 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:04.567313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee60ccc-eb4b-4d13-9950-adefa14c7152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:04.583895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cee60ccc-eb4b-4d13-9950-adefa14c7152 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:04.585434 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cee60ccc-eb4b-4d13-9950-adefa14c7152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:04.652056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 28/53] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:04.660277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff24e2c-96b6-4bd5-b99a-0ebfb26bfa88 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:04.677338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee60ccc-eb4b-4d13-9950-adefa14c7152 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:04.695630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239eabae-c4a6-418a-9b16-8beed61b8a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:04.696450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239eabae-c4a6-418a-9b16-8beed61b8a0b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:04.696874 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239eabae-c4a6-418a-9b16-8beed61b8a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:04.697153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239eabae-c4a6-418a-9b16-8beed61b8a0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:04.698469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239eabae-c4a6-418a-9b16-8beed61b8a0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:04.699939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239eabae-c4a6-418a-9b16-8beed61b8a0b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:04.701753 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 26/54] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:04.707855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239eabae-c4a6-418a-9b16-8beed61b8a0b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:04.758848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:04.767943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:04.778253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee60ccc-eb4b-4d13-9950-adefa14c7152 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:04.809189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239eabae-c4a6-418a-9b16-8beed61b8a0b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:04.812993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5658-c24c-4f13-92e8-b281df7e9604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:04.813804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5658-c24c-4f13-92e8-b281df7e9604 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:04.814122 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5658-c24c-4f13-92e8-b281df7e9604 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:04.814398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa5658-c24c-4f13-92e8-b281df7e9604 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:04.815603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa5658-c24c-4f13-92e8-b281df7e9604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:04.817089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa5658-c24c-4f13-92e8-b281df7e9604 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:04.818874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 29/55] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:04.824372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5658-c24c-4f13-92e8-b281df7e9604 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:04.843516 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07703c5-2572-4f3f-bf3b-c49b5834a21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:04.844229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07703c5-2572-4f3f-bf3b-c49b5834a21f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:04.844685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07703c5-2572-4f3f-bf3b-c49b5834a21f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:04.844956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07703c5-2572-4f3f-bf3b-c49b5834a21f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:04.845845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07703c5-2572-4f3f-bf3b-c49b5834a21f None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.domain.id=default, user.id=78c2f3799fc8459a9a6e05f14a552498) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:04.846210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07703c5-2572-4f3f-bf3b-c49b5834a21f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:04.847695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07703c5-2572-4f3f-bf3b-c49b5834a21f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:04.856123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 27/56] 10.222.0.26 () {56 vars in 1322 bytes} [Mon Jul 4 09:01:04 2022] GET /identity/v3/role_assignments?user.id=78c2f3799fc8459a9a6e05f14a552498&scope.domain.id=default&role.id=2302517080ac42578aa81c606dd0ff2f => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:04.935081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa5658-c24c-4f13-92e8-b281df7e9604 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:04.977049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4090e4c-dc12-4903-9604-8a6ab88619fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:04.977731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4090e4c-dc12-4903-9604-8a6ab88619fa None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:04.978051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4090e4c-dc12-4903-9604-8a6ab88619fa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:04.978341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4090e4c-dc12-4903-9604-8a6ab88619fa None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:04.979086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4090e4c-dc12-4903-9604-8a6ab88619fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:04.980559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4090e4c-dc12-4903-9604-8a6ab88619fa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:05.014121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 30/57] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Jul 4 09:01:04 2022] POST /identity/v3/projects => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:07.028979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37d5d0-f0e2-4d97-affd-c1f3e824c2be None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:07.029813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37d5d0-f0e2-4d97-affd-c1f3e824c2be None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:07.029978 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37d5d0-f0e2-4d97-affd-c1f3e824c2be None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:07.032093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 28/58] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:07.041405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2deca-e03a-4cb7-8f2e-5e2a2103ca64 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:07.041823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2deca-e03a-4cb7-8f2e-5e2a2103ca64 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:07.042098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2deca-e03a-4cb7-8f2e-5e2a2103ca64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:07.064128 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d3c2deca-e03a-4cb7-8f2e-5e2a2103ca64 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:07.065356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d3c2deca-e03a-4cb7-8f2e-5e2a2103ca64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:07.068222 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:07.137499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 31/59] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:07.162899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37d5d0-f0e2-4d97-affd-c1f3e824c2be None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:07.277662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37d5d0-f0e2-4d97-affd-c1f3e824c2be None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:07.292977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:07.315889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1af81e-88a0-4b0b-aad4-7913b787c773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:07.316692 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1af81e-88a0-4b0b-aad4-7913b787c773 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:07.316983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1af81e-88a0-4b0b-aad4-7913b787c773 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:07.317262 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1af81e-88a0-4b0b-aad4-7913b787c773 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:07.319098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1af81e-88a0-4b0b-aad4-7913b787c773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:07.320577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1af81e-88a0-4b0b-aad4-7913b787c773 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:07.322233 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 29/60] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:07.328028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2deca-e03a-4cb7-8f2e-5e2a2103ca64 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:07.431524 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2deca-e03a-4cb7-8f2e-5e2a2103ca64 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:07.468263 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b47eda-f295-483e-85e3-5e8f8f0f08bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:07.469266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b47eda-f295-483e-85e3-5e8f8f0f08bd None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:07.469709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b47eda-f295-483e-85e3-5e8f8f0f08bd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:07.470198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b47eda-f295-483e-85e3-5e8f8f0f08bd None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:07.471368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b47eda-f295-483e-85e3-5e8f8f0f08bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:07.473735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b47eda-f295-483e-85e3-5e8f8f0f08bd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:07.490516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:07.502414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 32/61] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:01:07 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:10.524621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58dcff-0893-4ad0-a7b6-550f1bb1d7b1 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:10.525510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58dcff-0893-4ad0-a7b6-550f1bb1d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:10.525993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58dcff-0893-4ad0-a7b6-550f1bb1d7b1 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:10.527796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 30/62] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:10.533353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67061047-2212-4ac5-8cb0-c76111e1815b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:10.534214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67061047-2212-4ac5-8cb0-c76111e1815b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:10.534694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67061047-2212-4ac5-8cb0-c76111e1815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:10.556298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-67061047-2212-4ac5-8cb0-c76111e1815b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:10.557855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-67061047-2212-4ac5-8cb0-c76111e1815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:10.630332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 33/63] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:10.667585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58dcff-0893-4ad0-a7b6-550f1bb1d7b1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:10.790735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58dcff-0893-4ad0-a7b6-550f1bb1d7b1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:10.835352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d77a4-825c-48ce-b310-e59de9cea92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:10.836253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d77a4-825c-48ce-b310-e59de9cea92c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:10.836535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d77a4-825c-48ce-b310-e59de9cea92c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:10.836808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d77a4-825c-48ce-b310-e59de9cea92c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:10.848666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d77a4-825c-48ce-b310-e59de9cea92c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ce6d77a4-825c-48ce-b310-e59de9cea92c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:10.857528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:10.858422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:10.859187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 31/64] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:10.866959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d77a4-825c-48ce-b310-e59de9cea92c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:10.867562 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a646366-8af4-46bf-a818-7ff5c33795c4 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:10.868233 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a646366-8af4-46bf-a818-7ff5c33795c4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:10.868714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a646366-8af4-46bf-a818-7ff5c33795c4 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:10.870554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 34/65] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:10.875844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de35fb-e4b2-4aed-b37e-6062579efe43 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:10.876366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de35fb-e4b2-4aed-b37e-6062579efe43 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:10.877289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de35fb-e4b2-4aed-b37e-6062579efe43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:10.918381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:10.920025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b0de35fb-e4b2-4aed-b37e-6062579efe43 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:10.921473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b0de35fb-e4b2-4aed-b37e-6062579efe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:10.970375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d77a4-825c-48ce-b310-e59de9cea92c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:11.007189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab8b93-4ff5-43d9-8a66-5f86d4641268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:11.008205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8b93-4ff5-43d9-8a66-5f86d4641268 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.008665 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8b93-4ff5-43d9-8a66-5f86d4641268 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.009095 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8b93-4ff5-43d9-8a66-5f86d4641268 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.010029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab8b93-4ff5-43d9-8a66-5f86d4641268 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:11.010507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab8b93-4ff5-43d9-8a66-5f86d4641268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:11.012054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab8b93-4ff5-43d9-8a66-5f86d4641268 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:11.028478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 32/66] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:11.030008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab8b93-4ff5-43d9-8a66-5f86d4641268 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:11.057014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 35/67] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:11.092366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de35fb-e4b2-4aed-b37e-6062579efe43 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:11.200344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:11.207217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab8b93-4ff5-43d9-8a66-5f86d4641268 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:11.210346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:11.242331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bdbd1-b8c1-4c6d-a89e-c87e54216add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:11.243508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bdbd1-b8c1-4c6d-a89e-c87e54216add None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.243989 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bdbd1-b8c1-4c6d-a89e-c87e54216add None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.244460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bdbd1-b8c1-4c6d-a89e-c87e54216add None admin] PATH_INFO: `/v3/users/264b2fc6044940aba215d38e72c6e93a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.250114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de35fb-e4b2-4aed-b37e-6062579efe43 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:11.260959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03bdbd1-b8c1-4c6d-a89e-c87e54216add None admin] RBAC: Authorizing `identity:get_user(user_id=264b2fc6044940aba215d38e72c6e93a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:11.262744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03bdbd1-b8c1-4c6d-a89e-c87e54216add None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:11.265039 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 33/68] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:11 2022] GET /identity/v3/users/264b2fc6044940aba215d38e72c6e93a => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:11.266585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8d3fe-c7c2-4c2c-8b1f-9584daf30c4d None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.267083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8d3fe-c7c2-4c2c-8b1f-9584daf30c4d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.267550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8d3fe-c7c2-4c2c-8b1f-9584daf30c4d None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.269097 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 34/69] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:11.270063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8d3fe-c7c2-4c2c-8b1f-9584daf30c4d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:11.291254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9e4b-21e7-451e-b2a9-a7bed5cc2511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:11.292067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9e4b-21e7-451e-b2a9-a7bed5cc2511 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.292357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9e4b-21e7-451e-b2a9-a7bed5cc2511 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.292640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9e4b-21e7-451e-b2a9-a7bed5cc2511 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.304565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf9e4b-21e7-451e-b2a9-a7bed5cc2511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-90bf9e4b-21e7-451e-b2a9-a7bed5cc2511 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:11.313097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:11.314095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:11.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:11.317115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 36/70] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:01:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:11.318641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93afa0e-6af7-4155-8776-6a1a221ece0d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.319099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93afa0e-6af7-4155-8776-6a1a221ece0d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.319564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93afa0e-6af7-4155-8776-6a1a221ece0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.337809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b93afa0e-6af7-4155-8776-6a1a221ece0d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:11.339046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b93afa0e-6af7-4155-8776-6a1a221ece0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:11.379502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8d3fe-c7c2-4c2c-8b1f-9584daf30c4d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:11.410042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 37/71] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:11.410042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93afa0e-6af7-4155-8776-6a1a221ece0d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:11.420362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7648b2-0e53-4f04-8c33-aacd78637b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:11.421155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7648b2-0e53-4f04-8c33-aacd78637b57 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.421426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7648b2-0e53-4f04-8c33-aacd78637b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.421704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7648b2-0e53-4f04-8c33-aacd78637b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.454085 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:11.464989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:11.495626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7648b2-0e53-4f04-8c33-aacd78637b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:11.497124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7648b2-0e53-4f04-8c33-aacd78637b57 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:11.522093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93afa0e-6af7-4155-8776-6a1a221ece0d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:11.556645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be942de-56c1-4b6a-87f9-4ecf705ae46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:11.557313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be942de-56c1-4b6a-87f9-4ecf705ae46b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.557634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be942de-56c1-4b6a-87f9-4ecf705ae46b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.557952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be942de-56c1-4b6a-87f9-4ecf705ae46b None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.558742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be942de-56c1-4b6a-87f9-4ecf705ae46b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:11.559043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be942de-56c1-4b6a-87f9-4ecf705ae46b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:11.560518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be942de-56c1-4b6a-87f9-4ecf705ae46b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:11.569101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 38/72] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:11.570358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36c5d2-a056-4827-8738-8c84a3312e77 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.570654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36c5d2-a056-4827-8738-8c84a3312e77 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.570939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36c5d2-a056-4827-8738-8c84a3312e77 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.572329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 39/73] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:11.572892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36c5d2-a056-4827-8738-8c84a3312e77 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:11.601320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 35/74] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:11.603326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd63cc-0a54-41c5-b472-906506e7160c None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.603623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd63cc-0a54-41c5-b472-906506e7160c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.603897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd63cc-0a54-41c5-b472-906506e7160c None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.605196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 36/75] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:11.610743 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd63cc-0a54-41c5-b472-906506e7160c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:11.674443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:11.681391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36c5d2-a056-4827-8738-8c84a3312e77 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:11.690340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:11.714624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1234c7-ce6f-4bc0-9cd0-494caa0074ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:11.715436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1234c7-ce6f-4bc0-9cd0-494caa0074ef None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.715720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1234c7-ce6f-4bc0-9cd0-494caa0074ef None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.715996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1234c7-ce6f-4bc0-9cd0-494caa0074ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.716427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd63cc-0a54-41c5-b472-906506e7160c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:11.732085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1234c7-ce6f-4bc0-9cd0-494caa0074ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-0e1234c7-ce6f-4bc0-9cd0-494caa0074ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:11.744889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:11.745999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:11.746871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 40/76] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:11.747518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f489b1b-7299-42fc-8c7a-36da3dc92e61 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.748087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f489b1b-7299-42fc-8c7a-36da3dc92e61 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.748087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f489b1b-7299-42fc-8c7a-36da3dc92e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.763585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00038a9c-d506-4460-8c41-cb5597d3c19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:11.764456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00038a9c-d506-4460-8c41-cb5597d3c19d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.765962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00038a9c-d506-4460-8c41-cb5597d3c19d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.766253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00038a9c-d506-4460-8c41-cb5597d3c19d None admin] PATH_INFO: `/v3/users/78c2f3799fc8459a9a6e05f14a552498` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.768868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0f489b1b-7299-42fc-8c7a-36da3dc92e61 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:11.768970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00038a9c-d506-4460-8c41-cb5597d3c19d None admin] RBAC: Authorizing `identity:get_user(user_id=78c2f3799fc8459a9a6e05f14a552498)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:11.770141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0f489b1b-7299-42fc-8c7a-36da3dc92e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:11.770593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00038a9c-d506-4460-8c41-cb5597d3c19d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:11.772495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 37/77] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:11 2022] GET /identity/v3/users/78c2f3799fc8459a9a6e05f14a552498 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:11.773689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00038a9c-d506-4460-8c41-cb5597d3c19d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:11.857898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 41/78] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:11.859675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4c9a3-d8b4-42fc-8290-b264bc52757d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.860193 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4c9a3-d8b4-42fc-8290-b264bc52757d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.860579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4c9a3-d8b4-42fc-8290-b264bc52757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.882004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-30d4c9a3-d8b4-42fc-8290-b264bc52757d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:11.882004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-30d4c9a3-d8b4-42fc-8290-b264bc52757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:11.901219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00038a9c-d506-4460-8c41-cb5597d3c19d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:11.939045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:11.950294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1b3f3-4fae-4d82-aedb-28724af26292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:11.951191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1b3f3-4fae-4d82-aedb-28724af26292 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:11.951479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1b3f3-4fae-4d82-aedb-28724af26292 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:11.951761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1b3f3-4fae-4d82-aedb-28724af26292 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:11.953039 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1b3f3-4fae-4d82-aedb-28724af26292 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:11.954571 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1b3f3-4fae-4d82-aedb-28724af26292 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:11.956373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 38/79] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:11 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:11.957075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1b3f3-4fae-4d82-aedb-28724af26292 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:11.970223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 42/80] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:11.970223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4c9a3-d8b4-42fc-8290-b264bc52757d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:11.999941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:12.074064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1b3f3-4fae-4d82-aedb-28724af26292 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:12.084357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4c9a3-d8b4-42fc-8290-b264bc52757d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:12.113787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223ca1e-9d8d-4d9c-a7d3-6cfa151f87a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:12.114694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ca1e-9d8d-4d9c-a7d3-6cfa151f87a8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:12.114991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ca1e-9d8d-4d9c-a7d3-6cfa151f87a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:12.115264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ca1e-9d8d-4d9c-a7d3-6cfa151f87a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:12.116330 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223ca1e-9d8d-4d9c-a7d3-6cfa151f87a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:12.116641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223ca1e-9d8d-4d9c-a7d3-6cfa151f87a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:12.118077 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223ca1e-9d8d-4d9c-a7d3-6cfa151f87a8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:12.123212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e4836-eda0-4057-b615-e342542f520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:12.123901 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e4836-eda0-4057-b615-e342542f520e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:12.124191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e4836-eda0-4057-b615-e342542f520e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:12.124478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e4836-eda0-4057-b615-e342542f520e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:12.132083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 39/81] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:12.132083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223ca1e-9d8d-4d9c-a7d3-6cfa151f87a8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:12.195661 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:12.218368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e4836-eda0-4057-b615-e342542f520e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:12.220529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e4836-eda0-4057-b615-e342542f520e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:12.243905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223ca1e-9d8d-4d9c-a7d3-6cfa151f87a8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:12.261358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:12.288814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd54a26-e7ab-4d7a-b176-0a4a565fcb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:12.289663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54a26-e7ab-4d7a-b176-0a4a565fcb99 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:12.289979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54a26-e7ab-4d7a-b176-0a4a565fcb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:12.290280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd54a26-e7ab-4d7a-b176-0a4a565fcb99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:12.301203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd54a26-e7ab-4d7a-b176-0a4a565fcb99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-bdd54a26-e7ab-4d7a-b176-0a4a565fcb99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:12.309073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:12.310117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 40/82] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:12.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd54a26-e7ab-4d7a-b176-0a4a565fcb99 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:12.350050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 43/83] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:12.351052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e4836-eda0-4057-b615-e342542f520e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:12.440186 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd54a26-e7ab-4d7a-b176-0a4a565fcb99 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:12.464185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:12.464815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e4836-eda0-4057-b615-e342542f520e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:12.479786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfa8ac-a96a-4104-abd9-f16453bbda9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:12.480614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfa8ac-a96a-4104-abd9-f16453bbda9a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:12.481090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfa8ac-a96a-4104-abd9-f16453bbda9a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:12.481547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfa8ac-a96a-4104-abd9-f16453bbda9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:12.482712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfa8ac-a96a-4104-abd9-f16453bbda9a None admin] RBAC: Adding query filter params (role.id=0b98291295724747acd01dd0d2306e62, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51, user.id=264b2fc6044940aba215d38e72c6e93a) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:12.483028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfa8ac-a96a-4104-abd9-f16453bbda9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:12.484526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfa8ac-a96a-4104-abd9-f16453bbda9a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:12.504757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 41/84] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:12 2022] GET /identity/v3/role_assignments?user.id=264b2fc6044940aba215d38e72c6e93a&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=0b98291295724747acd01dd0d2306e62 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:12.504757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfa8ac-a96a-4104-abd9-f16453bbda9a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:12.505027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6e6c4-377a-490e-afd8-a949d41ce56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:12.505027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e6c4-377a-490e-afd8-a949d41ce56d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:12.505027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e6c4-377a-490e-afd8-a949d41ce56d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:12.505027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e6c4-377a-490e-afd8-a949d41ce56d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:12.505507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6e6c4-377a-490e-afd8-a949d41ce56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:12.507184 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6e6c4-377a-490e-afd8-a949d41ce56d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:12.508977 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 44/85] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:12.509475 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:12.511645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6e6c4-377a-490e-afd8-a949d41ce56d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:12.620754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfa8ac-a96a-4104-abd9-f16453bbda9a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:12.636118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6e6c4-377a-490e-afd8-a949d41ce56d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:12.659811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf1625-515c-4efd-9a59-6295a5ba54c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:12.660708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf1625-515c-4efd-9a59-6295a5ba54c3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:12.661012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf1625-515c-4efd-9a59-6295a5ba54c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:12.661370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf1625-515c-4efd-9a59-6295a5ba54c3 None admin] PATH_INFO: `/v3/users/264b2fc6044940aba215d38e72c6e93a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:12.662646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf1625-515c-4efd-9a59-6295a5ba54c3 None admin] RBAC: Authorizing `identity:get_user(user_id=264b2fc6044940aba215d38e72c6e93a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:12.664263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf1625-515c-4efd-9a59-6295a5ba54c3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:12.665956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 42/86] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:12 2022] GET /identity/v3/users/264b2fc6044940aba215d38e72c6e93a => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:12.667598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf1625-515c-4efd-9a59-6295a5ba54c3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:12.682039 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c88866-971e-4344-86b3-388dd73e4d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:12.682819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c88866-971e-4344-86b3-388dd73e4d7d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:12.683200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c88866-971e-4344-86b3-388dd73e4d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:12.683536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c88866-971e-4344-86b3-388dd73e4d7d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:12.684343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c88866-971e-4344-86b3-388dd73e4d7d None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:12.684680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c88866-971e-4344-86b3-388dd73e4d7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:12.686108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c88866-971e-4344-86b3-388dd73e4d7d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:12.695240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 45/87] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:12.695976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c88866-971e-4344-86b3-388dd73e4d7d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:12.731985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:12.758123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:12.783288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf1625-515c-4efd-9a59-6295a5ba54c3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:12.801694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c88866-971e-4344-86b3-388dd73e4d7d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:12.826860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e23a92-36ea-4965-8cb7-f803269da834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:12.827703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e23a92-36ea-4965-8cb7-f803269da834 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:12.828107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e23a92-36ea-4965-8cb7-f803269da834 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:12.828386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e23a92-36ea-4965-8cb7-f803269da834 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:12.829946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e23a92-36ea-4965-8cb7-f803269da834 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:12.831378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e23a92-36ea-4965-8cb7-f803269da834 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:12.837897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 43/88] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:12 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:12.837897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e23a92-36ea-4965-8cb7-f803269da834 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:12.842803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd75dcf-3081-4dc2-9963-a9f7b235b727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:12.843500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd75dcf-3081-4dc2-9963-a9f7b235b727 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:12.843794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd75dcf-3081-4dc2-9963-a9f7b235b727 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:12.844082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd75dcf-3081-4dc2-9963-a9f7b235b727 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:12.844823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd75dcf-3081-4dc2-9963-a9f7b235b727 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:12.846341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd75dcf-3081-4dc2-9963-a9f7b235b727 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:12.870948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 46/89] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:01:12 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:12.871879 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd75dcf-3081-4dc2-9963-a9f7b235b727 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:12.989837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e23a92-36ea-4965-8cb7-f803269da834 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:13.013967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd75dcf-3081-4dc2-9963-a9f7b235b727 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:13.018333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:13.030991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:13.038253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb37738-0585-457f-a0d9-4da91ae0c505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:13.039925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb37738-0585-457f-a0d9-4da91ae0c505 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:13.040553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb37738-0585-457f-a0d9-4da91ae0c505 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:13.041029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb37738-0585-457f-a0d9-4da91ae0c505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:13.056432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebebc5-bd84-4fd9-b1e9-3f25ecbcbd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:13.057358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebebc5-bd84-4fd9-b1e9-3f25ecbcbd1b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:13.057748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebebc5-bd84-4fd9-b1e9-3f25ecbcbd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:13.059658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebebc5-bd84-4fd9-b1e9-3f25ecbcbd1b None admin] PATH_INFO: `/v3/users/264b2fc6044940aba215d38e72c6e93a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:13.059973 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebebc5-bd84-4fd9-b1e9-3f25ecbcbd1b None admin] RBAC: Authorizing `identity:get_user(user_id=264b2fc6044940aba215d38e72c6e93a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:13.061459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebebc5-bd84-4fd9-b1e9-3f25ecbcbd1b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:13.063147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 47/90] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:12 2022] GET /identity/v3/users/264b2fc6044940aba215d38e72c6e93a => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:13.063926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebebc5-bd84-4fd9-b1e9-3f25ecbcbd1b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:13.135880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb37738-0585-457f-a0d9-4da91ae0c505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:13.137901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb37738-0585-457f-a0d9-4da91ae0c505 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:13.209292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebebc5-bd84-4fd9-b1e9-3f25ecbcbd1b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:13.257243 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f7052-f4a7-4707-8b38-1a37c0cca849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:13.257836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f7052-f4a7-4707-8b38-1a37c0cca849 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:13.258455 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f7052-f4a7-4707-8b38-1a37c0cca849 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:13.258744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f7052-f4a7-4707-8b38-1a37c0cca849 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:13.260066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f7052-f4a7-4707-8b38-1a37c0cca849 None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51, user.id=78c2f3799fc8459a9a6e05f14a552498) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:13.260534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f7052-f4a7-4707-8b38-1a37c0cca849 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:13.262392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f7052-f4a7-4707-8b38-1a37c0cca849 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:13.266282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 44/91] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:13.267773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb37738-0585-457f-a0d9-4da91ae0c505 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:13.269234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 48/92] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:13 2022] GET /identity/v3/role_assignments?user.id=78c2f3799fc8459a9a6e05f14a552498&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=2302517080ac42578aa81c606dd0ff2f => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:13.275059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f7052-f4a7-4707-8b38-1a37c0cca849 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:13.297074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:13.349405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:13.385288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb37738-0585-457f-a0d9-4da91ae0c505 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:13.448852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08319839-bbe3-412d-af7b-b079e370e1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:13.448852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319839-bbe3-412d-af7b-b079e370e1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:13.448852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319839-bbe3-412d-af7b-b079e370e1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:13.448852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319839-bbe3-412d-af7b-b079e370e1a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:13.452276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f7052-f4a7-4707-8b38-1a37c0cca849 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:13.511275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf757a-5f8d-448e-a76b-dd94a0c56ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:13.512072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf757a-5f8d-448e-a76b-dd94a0c56ceb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:13.512356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf757a-5f8d-448e-a76b-dd94a0c56ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:13.512633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf757a-5f8d-448e-a76b-dd94a0c56ceb None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:13.514575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf757a-5f8d-448e-a76b-dd94a0c56ceb None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:13.516038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf757a-5f8d-448e-a76b-dd94a0c56ceb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:13.520633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 49/93] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:13 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:13.524374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf757a-5f8d-448e-a76b-dd94a0c56ceb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:13.554582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08319839-bbe3-412d-af7b-b079e370e1a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:13.556092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08319839-bbe3-412d-af7b-b079e370e1a3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:13.598246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:13.738182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 45/94] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:13.744470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf757a-5f8d-448e-a76b-dd94a0c56ceb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:13.744937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08319839-bbe3-412d-af7b-b079e370e1a3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:13.775721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:13.832129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ebd9-fab1-40fe-b3db-61cd317ed4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:13.832850 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ebd9-fab1-40fe-b3db-61cd317ed4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:13.833942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ebd9-fab1-40fe-b3db-61cd317ed4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:13.834277 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ebd9-fab1-40fe-b3db-61cd317ed4d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:13.835469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6ebd9-fab1-40fe-b3db-61cd317ed4d2 None admin] RBAC: Adding query filter params (role.id=2eea147ab8b1426294a08106410f6d45, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51, user.id=264b2fc6044940aba215d38e72c6e93a) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:13.835942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6ebd9-fab1-40fe-b3db-61cd317ed4d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:13.838747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6ebd9-fab1-40fe-b3db-61cd317ed4d2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:13.849492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 50/95] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:13 2022] GET /identity/v3/role_assignments?user.id=264b2fc6044940aba215d38e72c6e93a&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=2eea147ab8b1426294a08106410f6d45 => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:13.904373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08319839-bbe3-412d-af7b-b079e370e1a3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:13.933738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:13.951415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0b748-4666-437f-819b-c7d340530a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:13.952365 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0b748-4666-437f-819b-c7d340530a43 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:13.952657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0b748-4666-437f-819b-c7d340530a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:13.952942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0b748-4666-437f-819b-c7d340530a43 None admin] PATH_INFO: `/v3/projects/362499a7e2bf4afb8ebf317978db3c52` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:13.956029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0b748-4666-437f-819b-c7d340530a43 None admin] RBAC: Authorizing `identity:get_project(project_id=362499a7e2bf4afb8ebf317978db3c52)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:13.957518 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0b748-4666-437f-819b-c7d340530a43 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:13.959621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 46/96] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:13 2022] GET /identity/v3/projects/362499a7e2bf4afb8ebf317978db3c52 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:13.965677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ebd9-fab1-40fe-b3db-61cd317ed4d2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:14.088729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ebd9-fab1-40fe-b3db-61cd317ed4d2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:14.128893 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956f78f-7fcb-43fe-9c57-0ce785d96cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:14.130003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956f78f-7fcb-43fe-9c57-0ce785d96cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:14.130469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956f78f-7fcb-43fe-9c57-0ce785d96cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:14.130940 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956f78f-7fcb-43fe-9c57-0ce785d96cb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:14.131978 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956f78f-7fcb-43fe-9c57-0ce785d96cb2 None admin] RBAC: Adding query filter params (role.id=0b98291295724747acd01dd0d2306e62, scope.project.id=362499a7e2bf4afb8ebf317978db3c52, user.id=264b2fc6044940aba215d38e72c6e93a) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:14.132484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956f78f-7fcb-43fe-9c57-0ce785d96cb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:14.134169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956f78f-7fcb-43fe-9c57-0ce785d96cb2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:14.141117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 51/97] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:13 2022] GET /identity/v3/role_assignments?user.id=264b2fc6044940aba215d38e72c6e93a&scope.project.id=362499a7e2bf4afb8ebf317978db3c52&role.id=0b98291295724747acd01dd0d2306e62 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:15.063099 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d76e9-b062-4343-bbfd-d69628ab73ce None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:15.064155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d76e9-b062-4343-bbfd-d69628ab73ce None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:15.064637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d76e9-b062-4343-bbfd-d69628ab73ce None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:15.066280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 47/98] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:15.073585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a43a6-53f1-4afa-82a0-b2b7130adbea None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:15.074452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a43a6-53f1-4afa-82a0-b2b7130adbea None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:15.074954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a43a6-53f1-4afa-82a0-b2b7130adbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:15.097160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ed9a43a6-53f1-4afa-82a0-b2b7130adbea None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:15.098749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ed9a43a6-53f1-4afa-82a0-b2b7130adbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:15.144967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:15.188714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 52/99] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:15.223957 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d76e9-b062-4343-bbfd-d69628ab73ce None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:15.346543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d76e9-b062-4343-bbfd-d69628ab73ce None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:15.391785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d2db4-c975-49c2-8f4b-e1fbcd857506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:15.392947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d2db4-c975-49c2-8f4b-e1fbcd857506 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:15.393429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d2db4-c975-49c2-8f4b-e1fbcd857506 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:15.393950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d2db4-c975-49c2-8f4b-e1fbcd857506 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:15.414484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d2db4-c975-49c2-8f4b-e1fbcd857506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-113d2db4-c975-49c2-8f4b-e1fbcd857506 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:15.422592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:15.423452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:15.424231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:15.426667 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 48/100] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:15.433269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a43a6-53f1-4afa-82a0-b2b7130adbea None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:15.547742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a43a6-53f1-4afa-82a0-b2b7130adbea None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:15.583101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496349d3-c7a3-42ca-9b5c-13a80a7174ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:15.584272 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496349d3-c7a3-42ca-9b5c-13a80a7174ce None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:15.584753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496349d3-c7a3-42ca-9b5c-13a80a7174ce None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:15.585221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496349d3-c7a3-42ca-9b5c-13a80a7174ce None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:15.586228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496349d3-c7a3-42ca-9b5c-13a80a7174ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:15.586747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496349d3-c7a3-42ca-9b5c-13a80a7174ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:15.588407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496349d3-c7a3-42ca-9b5c-13a80a7174ce None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:15.598186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 53/101] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:15.604647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d2db4-c975-49c2-8f4b-e1fbcd857506 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:15.671691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:15.741940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d2db4-c975-49c2-8f4b-e1fbcd857506 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:15.786439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6586f20-f637-4da3-ae6d-a91542f52cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:15.787431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6586f20-f637-4da3-ae6d-a91542f52cdc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:15.787723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6586f20-f637-4da3-ae6d-a91542f52cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:15.788007 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6586f20-f637-4da3-ae6d-a91542f52cdc None admin] PATH_INFO: `/v3/users/264b2fc6044940aba215d38e72c6e93a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:15.789223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6586f20-f637-4da3-ae6d-a91542f52cdc None admin] RBAC: Authorizing `identity:get_user(user_id=264b2fc6044940aba215d38e72c6e93a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:15.790729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6586f20-f637-4da3-ae6d-a91542f52cdc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:15.792672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 49/102] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:15 2022] GET /identity/v3/users/264b2fc6044940aba215d38e72c6e93a => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:15.798093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496349d3-c7a3-42ca-9b5c-13a80a7174ce None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:15.812540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d23e2-e948-4840-9e2a-969ed361d7bd None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:15.812834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d23e2-e948-4840-9e2a-969ed361d7bd None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:15.813126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d23e2-e948-4840-9e2a-969ed361d7bd None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:15.814683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 50/103] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:15.823399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e35342-59a8-4483-9a19-d30b817242c8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:15.823689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e35342-59a8-4483-9a19-d30b817242c8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:15.823969 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e35342-59a8-4483-9a19-d30b817242c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:15.863517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-51e35342-59a8-4483-9a19-d30b817242c8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:15.864732 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-51e35342-59a8-4483-9a19-d30b817242c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:15.883439 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:16.002727 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 51/104] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:16.010876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496349d3-c7a3-42ca-9b5c-13a80a7174ce None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:16.058915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e35342-59a8-4483-9a19-d30b817242c8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:16.079308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14a089-2ffe-452a-a46f-4781d78f8e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:16.080339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14a089-2ffe-452a-a46f-4781d78f8e40 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:16.080608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14a089-2ffe-452a-a46f-4781d78f8e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:16.080892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad14a089-2ffe-452a-a46f-4781d78f8e40 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:16.086652 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14a089-2ffe-452a-a46f-4781d78f8e40 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:16.088153 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad14a089-2ffe-452a-a46f-4781d78f8e40 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:16.090239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 54/105] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:15 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:16.111888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14a089-2ffe-452a-a46f-4781d78f8e40 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:16.165200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:16.292551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e35342-59a8-4483-9a19-d30b817242c8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:16.324835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad14a089-2ffe-452a-a46f-4781d78f8e40 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:16.330187 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:16.362479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfb851-cb41-4bf5-b872-c33319794185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:16.363606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb851-cb41-4bf5-b872-c33319794185 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:16.363898 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb851-cb41-4bf5-b872-c33319794185 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:16.364210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb851-cb41-4bf5-b872-c33319794185 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:16.387023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfb851-cb41-4bf5-b872-c33319794185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-28bfb851-cb41-4bf5-b872-c33319794185 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:16.395881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:16.396764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:16.397982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 52/106] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:16.407181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfb851-cb41-4bf5-b872-c33319794185 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:16.410622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09006a-e24d-47f4-b786-0fb57c10b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:16.411913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09006a-e24d-47f4-b786-0fb57c10b36b None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:16.412423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09006a-e24d-47f4-b786-0fb57c10b36b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:16.412892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09006a-e24d-47f4-b786-0fb57c10b36b None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/users/264b2fc6044940aba215d38e72c6e93a/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:16.419517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09006a-e24d-47f4-b786-0fb57c10b36b None admin] RBAC: Authorizing `identity:create_grant(project_id=795c9a5eff6345f6ae2de8e24f361e51, user_id=264b2fc6044940aba215d38e72c6e93a, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:16.421358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09006a-e24d-47f4-b786-0fb57c10b36b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:16.454240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 55/107] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:01:16 2022] PUT /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/users/264b2fc6044940aba215d38e72c6e93a/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:01:16.572833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfb851-cb41-4bf5-b872-c33319794185 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:16.583630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:16.616743 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e705a0-479e-4da8-b8a1-09a3f6d56577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:16.617448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e705a0-479e-4da8-b8a1-09a3f6d56577 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:16.617737 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e705a0-479e-4da8-b8a1-09a3f6d56577 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:16.618054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e705a0-479e-4da8-b8a1-09a3f6d56577 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:16.618836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e705a0-479e-4da8-b8a1-09a3f6d56577 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:16.619150 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e705a0-479e-4da8-b8a1-09a3f6d56577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:16.620579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e705a0-479e-4da8-b8a1-09a3f6d56577 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:16.634007 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 53/108] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:16.639889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09006a-e24d-47f4-b786-0fb57c10b36b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:16.736563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affa25d-b0fc-4708-a0ca-88d34360677e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:16.736855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affa25d-b0fc-4708-a0ca-88d34360677e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:16.737045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affa25d-b0fc-4708-a0ca-88d34360677e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:16.738661 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 54/109] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:16.744998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d7b5e-8683-4e01-a080-ebaaaec9574f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:16.745846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d7b5e-8683-4e01-a080-ebaaaec9574f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:16.746627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d7b5e-8683-4e01-a080-ebaaaec9574f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:16.777647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-bb9d7b5e-8683-4e01-a080-ebaaaec9574f None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:16.781083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bb9d7b5e-8683-4e01-a080-ebaaaec9574f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:16.823348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09006a-e24d-47f4-b786-0fb57c10b36b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:16.871674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2a0f6-1f6d-4bbf-ab85-35cf3875bd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:16.872617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2a0f6-1f6d-4bbf-ab85-35cf3875bd94 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:16.872902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2a0f6-1f6d-4bbf-ab85-35cf3875bd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:16.873226 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2a0f6-1f6d-4bbf-ab85-35cf3875bd94 None admin] PATH_INFO: `/v3/users/0f69336f9fdb4a51b0f910df3f54a336` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:16.893351 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2a0f6-1f6d-4bbf-ab85-35cf3875bd94 None admin] RBAC: Authorizing `identity:get_user(user_id=0f69336f9fdb4a51b0f910df3f54a336)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:16.895552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2a0f6-1f6d-4bbf-ab85-35cf3875bd94 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:16.897639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 56/110] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:16 2022] GET /identity/v3/users/0f69336f9fdb4a51b0f910df3f54a336 => generated 289 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:16.898973 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a676c-000a-4c72-a4a4-1620559c6cc5 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:16.899273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a676c-000a-4c72-a4a4-1620559c6cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:16.899550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a676c-000a-4c72-a4a4-1620559c6cc5 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:16.900319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 55/111] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:16.900767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 57/112] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:16.906203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d7b5e-8683-4e01-a080-ebaaaec9574f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:16.914179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e170268-98c3-4161-9d9d-df8e8a8c7ba9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:16.914474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e170268-98c3-4161-9d9d-df8e8a8c7ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:16.914792 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e170268-98c3-4161-9d9d-df8e8a8c7ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:16.935696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2e170268-98c3-4161-9d9d-df8e8a8c7ba9 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:16.936915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2e170268-98c3-4161-9d9d-df8e8a8c7ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:16.956853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:17.025480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 58/113] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:17.027404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8517e6-a78b-4e43-90cd-3b412c5996e7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:17.028416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8517e6-a78b-4e43-90cd-3b412c5996e7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:17.028416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8517e6-a78b-4e43-90cd-3b412c5996e7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:17.029820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 59/114] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:17.030797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8517e6-a78b-4e43-90cd-3b412c5996e7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:17.042264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:17.045375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d7b5e-8683-4e01-a080-ebaaaec9574f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:17.084261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fd33d-6d4b-46eb-9cc9-1153baf3fbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:17.084974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fd33d-6d4b-46eb-9cc9-1153baf3fbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:17.085263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fd33d-6d4b-46eb-9cc9-1153baf3fbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:17.085590 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fd33d-6d4b-46eb-9cc9-1153baf3fbd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:17.146951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8517e6-a78b-4e43-90cd-3b412c5996e7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:17.164385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92fd33d-6d4b-46eb-9cc9-1153baf3fbd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:17.165915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92fd33d-6d4b-46eb-9cc9-1153baf3fbd9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:17.184547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea95d6-2b69-4bf5-9f4e-b475e2bbe351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:17.185357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea95d6-2b69-4bf5-9f4e-b475e2bbe351 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:17.185649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea95d6-2b69-4bf5-9f4e-b475e2bbe351 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:17.185955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea95d6-2b69-4bf5-9f4e-b475e2bbe351 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:17.187258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea95d6-2b69-4bf5-9f4e-b475e2bbe351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:17.188846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea95d6-2b69-4bf5-9f4e-b475e2bbe351 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:17.190812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 60/115] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:17.191935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889337f-c43c-4fea-817c-17d3779ad461 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:17.192398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889337f-c43c-4fea-817c-17d3779ad461 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:17.192703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889337f-c43c-4fea-817c-17d3779ad461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:17.209674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7889337f-c43c-4fea-817c-17d3779ad461 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:17.211023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7889337f-c43c-4fea-817c-17d3779ad461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:17.225322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:17.262256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:17.272836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 56/116] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:01:17.272836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f104a70-39bf-4824-8491-6fc2132b6454 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:17.272836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f104a70-39bf-4824-8491-6fc2132b6454 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:17.272836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f104a70-39bf-4824-8491-6fc2132b6454 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:17.272836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 57/117] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:17.272836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f104a70-39bf-4824-8491-6fc2132b6454 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:17.290016 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 61/118] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:17.290644 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889337f-c43c-4fea-817c-17d3779ad461 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:17.396952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889337f-c43c-4fea-817c-17d3779ad461 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:17.397747 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f104a70-39bf-4824-8491-6fc2132b6454 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:17.431946 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5951b58-6bc3-4311-984e-7aa95f15685f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:17.432623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5951b58-6bc3-4311-984e-7aa95f15685f None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:17.432909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5951b58-6bc3-4311-984e-7aa95f15685f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:17.433223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5951b58-6bc3-4311-984e-7aa95f15685f None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:17.433982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5951b58-6bc3-4311-984e-7aa95f15685f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:17.435433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5951b58-6bc3-4311-984e-7aa95f15685f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:17.442788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0125e-99ae-4563-aad9-7dc523773051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:17.444379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0125e-99ae-4563-aad9-7dc523773051 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:17.444886 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0125e-99ae-4563-aad9-7dc523773051 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:17.445692 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0125e-99ae-4563-aad9-7dc523773051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:17.456086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:17.457518 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0125e-99ae-4563-aad9-7dc523773051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:17.464400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 62/119] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:01:17 2022] POST /identity/v3/projects => generated 302 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:17.465205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5951b58-6bc3-4311-984e-7aa95f15685f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-bfb0125e-99ae-4563-aad9-7dc523773051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:17.465309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:17.466213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:17.467046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 58/120] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:01:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:17.468311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be1009-5670-4ec2-857d-575a24338cef None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:17.468311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be1009-5670-4ec2-857d-575a24338cef None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:17.468311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be1009-5670-4ec2-857d-575a24338cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:17.487582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-32be1009-5670-4ec2-857d-575a24338cef None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:17.488927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-32be1009-5670-4ec2-857d-575a24338cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:17.560460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:17.577611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 59/121] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:17.587476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be1009-5670-4ec2-857d-575a24338cef None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:17.631978 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5951b58-6bc3-4311-984e-7aa95f15685f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:17.677086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a699d9-165d-4129-bde4-46b840079831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:17.677086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a699d9-165d-4129-bde4-46b840079831 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:17.677494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a699d9-165d-4129-bde4-46b840079831 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:17.677793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a699d9-165d-4129-bde4-46b840079831 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:17.680840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a699d9-165d-4129-bde4-46b840079831 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:17.682712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a699d9-165d-4129-bde4-46b840079831 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:17.684806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 63/122] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:17 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:17.685659 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a699d9-165d-4129-bde4-46b840079831 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:17.746593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be1009-5670-4ec2-857d-575a24338cef None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:17.768622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:17.799312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cffe8-4d91-46c6-a283-a28e6bae3f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:17.801074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cffe8-4d91-46c6-a283-a28e6bae3f9e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:17.801735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cffe8-4d91-46c6-a283-a28e6bae3f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:17.802268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cffe8-4d91-46c6-a283-a28e6bae3f9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:17.818954 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5cffe8-4d91-46c6-a283-a28e6bae3f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:17.819845 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a699d9-165d-4129-bde4-46b840079831 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-7b5cffe8-4d91-46c6-a283-a28e6bae3f9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:17.834237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:17.835205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:17.835971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:17.838923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 60/123] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:17.840078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cffe8-4d91-46c6-a283-a28e6bae3f9e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:17.861834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a416783-130f-4ca8-b760-7c4b75b989c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:17.862573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a416783-130f-4ca8-b760-7c4b75b989c3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:17.862860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a416783-130f-4ca8-b760-7c4b75b989c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:17.863145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a416783-130f-4ca8-b760-7c4b75b989c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:17.863908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a416783-130f-4ca8-b760-7c4b75b989c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:17.864735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a416783-130f-4ca8-b760-7c4b75b989c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:17.866163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a416783-130f-4ca8-b760-7c4b75b989c3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:17.874928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 64/124] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:17.875977 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a416783-130f-4ca8-b760-7c4b75b989c3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:17.904711 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:17.968966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cffe8-4d91-46c6-a283-a28e6bae3f9e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:17.991394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a416783-130f-4ca8-b760-7c4b75b989c3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:18.018760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559d3f-3624-4ccf-81d2-acc1b7204cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:18.020057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559d3f-3624-4ccf-81d2-acc1b7204cef None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:18.020709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559d3f-3624-4ccf-81d2-acc1b7204cef None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:18.021189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559d3f-3624-4ccf-81d2-acc1b7204cef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:18.029767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:18.034527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf559d3f-3624-4ccf-81d2-acc1b7204cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:18.035868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193e14b-9878-4140-9c03-c459faef85d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:18.037119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e14b-9878-4140-9c03-c459faef85d5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:18.037750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e14b-9878-4140-9c03-c459faef85d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:18.038397 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e14b-9878-4140-9c03-c459faef85d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:18.039746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e14b-9878-4140-9c03-c459faef85d5 None admin] RBAC: Adding query filter params (role.id=67473bdffb674c6b8d0afd413642eff6, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51, user.id=0f69336f9fdb4a51b0f910df3f54a336) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:18.040226 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e14b-9878-4140-9c03-c459faef85d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:18.041953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e14b-9878-4140-9c03-c459faef85d5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-bf559d3f-3624-4ccf-81d2-acc1b7204cef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:18.045484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:18.046511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:18.047319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:18.049030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 65/125] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:17 2022] GET /identity/v3/role_assignments?user.id=0f69336f9fdb4a51b0f910df3f54a336&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=67473bdffb674c6b8d0afd413642eff6 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:18.049784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 61/126] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:18.050926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559d3f-3624-4ccf-81d2-acc1b7204cef None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:18.054791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193e14b-9878-4140-9c03-c459faef85d5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:18.253172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559d3f-3624-4ccf-81d2-acc1b7204cef None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:18.269059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193e14b-9878-4140-9c03-c459faef85d5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:18.280778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:18.308759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7c378-e454-4a95-85a6-5c8761152464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:18.309744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c378-e454-4a95-85a6-5c8761152464 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:18.310264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c378-e454-4a95-85a6-5c8761152464 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:18.310721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c378-e454-4a95-85a6-5c8761152464 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:18.311861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7c378-e454-4a95-85a6-5c8761152464 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:18.312657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7c378-e454-4a95-85a6-5c8761152464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:18.314325 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7c378-e454-4a95-85a6-5c8761152464 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:18.329405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 66/127] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:18.332308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7c378-e454-4a95-85a6-5c8761152464 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:18.348291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25829b36-0c1a-4798-a850-a10f1be915f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:18.349720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25829b36-0c1a-4798-a850-a10f1be915f3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:18.350215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25829b36-0c1a-4798-a850-a10f1be915f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:18.350697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25829b36-0c1a-4798-a850-a10f1be915f3 None admin] PATH_INFO: `/v3/users/78c2f3799fc8459a9a6e05f14a552498` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:18.352452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25829b36-0c1a-4798-a850-a10f1be915f3 None admin] RBAC: Authorizing `identity:get_user(user_id=78c2f3799fc8459a9a6e05f14a552498)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:18.356179 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25829b36-0c1a-4798-a850-a10f1be915f3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:18.359470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 62/128] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:18 2022] GET /identity/v3/users/78c2f3799fc8459a9a6e05f14a552498 => generated 247 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:18.360429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25829b36-0c1a-4798-a850-a10f1be915f3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:18.472040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:18.486435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25829b36-0c1a-4798-a850-a10f1be915f3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:18.524435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38d647-1208-429d-be0b-4bdf209f03ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:18.525793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38d647-1208-429d-be0b-4bdf209f03ef None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:18.526299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38d647-1208-429d-be0b-4bdf209f03ef None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:18.526769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38d647-1208-429d-be0b-4bdf209f03ef None admin] PATH_INFO: `/v3/users/264b2fc6044940aba215d38e72c6e93a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:18.528298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38d647-1208-429d-be0b-4bdf209f03ef None admin] RBAC: Authorizing `identity:get_user(user_id=264b2fc6044940aba215d38e72c6e93a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:18.530075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38d647-1208-429d-be0b-4bdf209f03ef None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:18.532243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 63/129] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:18 2022] GET /identity/v3/users/264b2fc6044940aba215d38e72c6e93a => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:18.533191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38d647-1208-429d-be0b-4bdf209f03ef None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:18.542344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7c378-e454-4a95-85a6-5c8761152464 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:18.579785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc0d42-b13a-4529-8765-bf9968ee790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:18.580472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc0d42-b13a-4529-8765-bf9968ee790a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:18.580760 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc0d42-b13a-4529-8765-bf9968ee790a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:18.581040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc0d42-b13a-4529-8765-bf9968ee790a None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:18.581982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc0d42-b13a-4529-8765-bf9968ee790a None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:18.582266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc0d42-b13a-4529-8765-bf9968ee790a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:18.583707 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc0d42-b13a-4529-8765-bf9968ee790a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:18.585297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:18.593691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 67/130] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:18.595214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc0d42-b13a-4529-8765-bf9968ee790a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:18.646270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38d647-1208-429d-be0b-4bdf209f03ef None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:18.682392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8fda1-e17d-41c4-992a-bfae62f9a5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:18.683681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8fda1-e17d-41c4-992a-bfae62f9a5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:18.684173 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8fda1-e17d-41c4-992a-bfae62f9a5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:18.684636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8fda1-e17d-41c4-992a-bfae62f9a5c1 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:18.686143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8fda1-e17d-41c4-992a-bfae62f9a5c1 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:18.687818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8fda1-e17d-41c4-992a-bfae62f9a5c1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:18.689726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 64/131] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:18 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:18.690718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8fda1-e17d-41c4-992a-bfae62f9a5c1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:18.789012 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc0d42-b13a-4529-8765-bf9968ee790a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:18.796479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8fda1-e17d-41c4-992a-bfae62f9a5c1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:18.810238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:18.838839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca51ac-0198-4145-a6a7-b0f3d5eb6358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:18.839620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca51ac-0198-4145-a6a7-b0f3d5eb6358 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:18.840701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca51ac-0198-4145-a6a7-b0f3d5eb6358 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:18.840701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca51ac-0198-4145-a6a7-b0f3d5eb6358 None admin] PATH_INFO: `/v3/users/264b2fc6044940aba215d38e72c6e93a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:18.841353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca51ac-0198-4145-a6a7-b0f3d5eb6358 None admin] RBAC: Authorizing `identity:get_user(user_id=264b2fc6044940aba215d38e72c6e93a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:18.844951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:18.845063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca51ac-0198-4145-a6a7-b0f3d5eb6358 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:18.849862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 65/132] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:18 2022] GET /identity/v3/users/264b2fc6044940aba215d38e72c6e93a => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:18.855705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca51ac-0198-4145-a6a7-b0f3d5eb6358 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:18.880693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358fde1-fbaf-4dd1-8941-403f0020d1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:18.881681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358fde1-fbaf-4dd1-8941-403f0020d1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:18.882006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358fde1-fbaf-4dd1-8941-403f0020d1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:18.882293 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358fde1-fbaf-4dd1-8941-403f0020d1c4 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:18.883638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358fde1-fbaf-4dd1-8941-403f0020d1c4 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:18.885115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358fde1-fbaf-4dd1-8941-403f0020d1c4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:18.887031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 68/133] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:18 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:18.889353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358fde1-fbaf-4dd1-8941-403f0020d1c4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:19.033492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca51ac-0198-4145-a6a7-b0f3d5eb6358 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:19.088407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358fde1-fbaf-4dd1-8941-403f0020d1c4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:19.095508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83010f61-01fb-49a3-bbc9-986dab6f2e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:19.096472 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010f61-01fb-49a3-bbc9-986dab6f2e35 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:19.096804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010f61-01fb-49a3-bbc9-986dab6f2e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:19.097094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010f61-01fb-49a3-bbc9-986dab6f2e35 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/users/78c2f3799fc8459a9a6e05f14a552498/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:19.099416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83010f61-01fb-49a3-bbc9-986dab6f2e35 None admin] RBAC: Authorizing `identity:create_grant(project_id=795c9a5eff6345f6ae2de8e24f361e51, user_id=78c2f3799fc8459a9a6e05f14a552498, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:19.100897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83010f61-01fb-49a3-bbc9-986dab6f2e35 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:19.119612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 66/134] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:01:18 2022] PUT /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/users/78c2f3799fc8459a9a6e05f14a552498/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:19.127623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83010f61-01fb-49a3-bbc9-986dab6f2e35 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:19.155941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab25865-3162-4a1a-8966-5063af7861cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:19.158763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab25865-3162-4a1a-8966-5063af7861cd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:19.159153 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab25865-3162-4a1a-8966-5063af7861cd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:19.159448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab25865-3162-4a1a-8966-5063af7861cd None admin] PATH_INFO: `/v3/projects/362499a7e2bf4afb8ebf317978db3c52` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:19.162903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab25865-3162-4a1a-8966-5063af7861cd None admin] RBAC: Authorizing `identity:get_project(project_id=362499a7e2bf4afb8ebf317978db3c52)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:19.168284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab25865-3162-4a1a-8966-5063af7861cd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:19.172680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 69/135] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:18 2022] GET /identity/v3/projects/362499a7e2bf4afb8ebf317978db3c52 => generated 312 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:19.174117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f582f6-b7ff-46b1-a828-5bed5e160d6d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:19.177410 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f582f6-b7ff-46b1-a828-5bed5e160d6d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:19.177716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f582f6-b7ff-46b1-a828-5bed5e160d6d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:19.181150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 70/136] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:19.190151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f582f6-b7ff-46b1-a828-5bed5e160d6d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:19.238123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:19.314898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:19.350762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83010f61-01fb-49a3-bbc9-986dab6f2e35 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:19.399317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f582f6-b7ff-46b1-a828-5bed5e160d6d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:19.404763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec6391-16ce-47c7-b363-f10e036910bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:19.405676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec6391-16ce-47c7-b363-f10e036910bf None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:19.405985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec6391-16ce-47c7-b363-f10e036910bf None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:19.406269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec6391-16ce-47c7-b363-f10e036910bf None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/users/264b2fc6044940aba215d38e72c6e93a/roles/2eea147ab8b1426294a08106410f6d45` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:19.413908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec6391-16ce-47c7-b363-f10e036910bf None admin] RBAC: Authorizing `identity:create_grant(project_id=795c9a5eff6345f6ae2de8e24f361e51, user_id=264b2fc6044940aba215d38e72c6e93a, role_id=2eea147ab8b1426294a08106410f6d45)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:19.415400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec6391-16ce-47c7-b363-f10e036910bf None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:19.445296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 67/137] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:01:19 2022] PUT /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/users/264b2fc6044940aba215d38e72c6e93a/roles/2eea147ab8b1426294a08106410f6d45 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:19.446718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa893c-a277-4dfe-8a7b-74f06a30674a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:19.447011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa893c-a277-4dfe-8a7b-74f06a30674a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:19.447294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa893c-a277-4dfe-8a7b-74f06a30674a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:19.467489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1283d5-39b8-4353-b05d-9c735ef1b998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:19.468775 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1283d5-39b8-4353-b05d-9c735ef1b998 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:19.469315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1283d5-39b8-4353-b05d-9c735ef1b998 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:19.469780 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1283d5-39b8-4353-b05d-9c735ef1b998 None admin] PATH_INFO: `/v3/projects/362499a7e2bf4afb8ebf317978db3c52/users/264b2fc6044940aba215d38e72c6e93a/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:19.472045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1283d5-39b8-4353-b05d-9c735ef1b998 None admin] RBAC: Authorizing `identity:create_grant(project_id=362499a7e2bf4afb8ebf317978db3c52, user_id=264b2fc6044940aba215d38e72c6e93a, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:19.473871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1283d5-39b8-4353-b05d-9c735ef1b998 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:19.481950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-08aa893c-a277-4dfe-8a7b-74f06a30674a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:19.481950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-08aa893c-a277-4dfe-8a7b-74f06a30674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:19.510425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 71/138] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:01:19 2022] PUT /identity/v3/projects/362499a7e2bf4afb8ebf317978db3c52/users/264b2fc6044940aba215d38e72c6e93a/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:19.569870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 68/139] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:19.605273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1283d5-39b8-4353-b05d-9c735ef1b998 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:19.744336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1283d5-39b8-4353-b05d-9c735ef1b998 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:19.765956 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:19.792993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a4b8f-abd1-498c-b80c-08d1c96f1013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:19.793812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a4b8f-abd1-498c-b80c-08d1c96f1013 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:19.794140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a4b8f-abd1-498c-b80c-08d1c96f1013 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:19.794423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a4b8f-abd1-498c-b80c-08d1c96f1013 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:19.808424 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a4b8f-abd1-498c-b80c-08d1c96f1013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-3c6a4b8f-abd1-498c-b80c-08d1c96f1013 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:19.816375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:19.817530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:19.818457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:19.819100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 72/140] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:19.827551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa893c-a277-4dfe-8a7b-74f06a30674a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:19.857012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:19.942707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa893c-a277-4dfe-8a7b-74f06a30674a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:19.946693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec692d5f-161d-43db-95a7-aaf8949e2313 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:19.947237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec692d5f-161d-43db-95a7-aaf8949e2313 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:19.947717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec692d5f-161d-43db-95a7-aaf8949e2313 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:19.949318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 73/141] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:19.954850 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c3795-2010-44f5-8dfb-0c0b17385839 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:19.955421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c3795-2010-44f5-8dfb-0c0b17385839 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:19.955895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c3795-2010-44f5-8dfb-0c0b17385839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:19.974826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-582c3795-2010-44f5-8dfb-0c0b17385839 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:19.976423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-582c3795-2010-44f5-8dfb-0c0b17385839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:19.991505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e2a6a-4fbb-4ab6-b583-4d33a9739dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:19.992205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e2a6a-4fbb-4ab6-b583-4d33a9739dbc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:19.992488 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e2a6a-4fbb-4ab6-b583-4d33a9739dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:19.992766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e2a6a-4fbb-4ab6-b583-4d33a9739dbc None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:19.993544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e2a6a-4fbb-4ab6-b583-4d33a9739dbc None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:19.993850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e2a6a-4fbb-4ab6-b583-4d33a9739dbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:19.995408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e2a6a-4fbb-4ab6-b583-4d33a9739dbc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:20.009790 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 69/142] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:20.015482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e2a6a-4fbb-4ab6-b583-4d33a9739dbc None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:20.065054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 74/143] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:20.106421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c3795-2010-44f5-8dfb-0c0b17385839 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:20.175929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:20.195984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e2a6a-4fbb-4ab6-b583-4d33a9739dbc None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:20.270740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668c41b-393f-4ce6-90f5-19037ad8c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:20.271815 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668c41b-393f-4ce6-90f5-19037ad8c331 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:20.272106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668c41b-393f-4ce6-90f5-19037ad8c331 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:20.272386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668c41b-393f-4ce6-90f5-19037ad8c331 None admin] PATH_INFO: `/v3/users/264b2fc6044940aba215d38e72c6e93a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:20.273664 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668c41b-393f-4ce6-90f5-19037ad8c331 None admin] RBAC: Authorizing `identity:get_user(user_id=264b2fc6044940aba215d38e72c6e93a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:20.275203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668c41b-393f-4ce6-90f5-19037ad8c331 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:20.282172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 70/144] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:20 2022] GET /identity/v3/users/264b2fc6044940aba215d38e72c6e93a => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:20.290930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668c41b-393f-4ce6-90f5-19037ad8c331 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:20.308709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c3795-2010-44f5-8dfb-0c0b17385839 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:20.342304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:20.354594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d2788-689d-4600-9baa-6e4f4a36283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:20.355430 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2788-689d-4600-9baa-6e4f4a36283f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:20.355713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2788-689d-4600-9baa-6e4f4a36283f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:20.355989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2788-689d-4600-9baa-6e4f4a36283f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:20.357218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d2788-689d-4600-9baa-6e4f4a36283f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:20.358743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d2788-689d-4600-9baa-6e4f4a36283f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:20.360392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 75/145] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:20.367132 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d2788-689d-4600-9baa-6e4f4a36283f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:20.495208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668c41b-393f-4ce6-90f5-19037ad8c331 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:20.552606 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d2788-689d-4600-9baa-6e4f4a36283f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:20.565060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5d752-4ff7-4ac4-9d4a-7bd2659d7304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:20.565736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5d752-4ff7-4ac4-9d4a-7bd2659d7304 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:20.566051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5d752-4ff7-4ac4-9d4a-7bd2659d7304 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:20.566334 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5d752-4ff7-4ac4-9d4a-7bd2659d7304 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:20.617666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05023557-4a20-4fca-98b9-9cb7c4e9e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:20.618390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023557-4a20-4fca-98b9-9cb7c4e9e9f2 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:20.618678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023557-4a20-4fca-98b9-9cb7c4e9e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:20.618954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023557-4a20-4fca-98b9-9cb7c4e9e9f2 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:20.619682 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05023557-4a20-4fca-98b9-9cb7c4e9e9f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:20.621147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05023557-4a20-4fca-98b9-9cb7c4e9e9f2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:20.627366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:20.672794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 76/146] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:01:20 2022] POST /identity/v3/users => generated 283 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:20.692651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5d752-4ff7-4ac4-9d4a-7bd2659d7304 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:20.694204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5d752-4ff7-4ac4-9d4a-7bd2659d7304 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:20.833609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 71/147] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:20.839842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05023557-4a20-4fca-98b9-9cb7c4e9e9f2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:20.877453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:20.880098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb26669-39b4-4b19-9ab0-77d90b689674 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:20.880602 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb26669-39b4-4b19-9ab0-77d90b689674 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:20.880891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb26669-39b4-4b19-9ab0-77d90b689674 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:20.882514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 72/148] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:20.888135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48eb23-f987-4bb4-8b25-d89b2c8e85e7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:20.888541 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48eb23-f987-4bb4-8b25-d89b2c8e85e7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:20.888709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48eb23-f987-4bb4-8b25-d89b2c8e85e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:20.913111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ee48eb23-f987-4bb4-8b25-d89b2c8e85e7 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:20.914417 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ee48eb23-f987-4bb4-8b25-d89b2c8e85e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:20.993670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:20.998963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05023557-4a20-4fca-98b9-9cb7c4e9e9f2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:21.014764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 73/149] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:21.048451 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455d0ff-4e28-4687-80ea-b707b61ec5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:21.048648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48eb23-f987-4bb4-8b25-d89b2c8e85e7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:21.049348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455d0ff-4e28-4687-80ea-b707b61ec5bb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:21.049632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455d0ff-4e28-4687-80ea-b707b61ec5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:21.049967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455d0ff-4e28-4687-80ea-b707b61ec5bb None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:21.051198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455d0ff-4e28-4687-80ea-b707b61ec5bb None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:21.053402 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455d0ff-4e28-4687-80ea-b707b61ec5bb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:21.055480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 77/150] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:20 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:21.060815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455d0ff-4e28-4687-80ea-b707b61ec5bb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:21.169631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:21.177526 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455d0ff-4e28-4687-80ea-b707b61ec5bb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:21.240926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48eb23-f987-4bb4-8b25-d89b2c8e85e7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:21.246504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6df6a-f22b-46f9-ad83-d016fa2536f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:21.247183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6df6a-f22b-46f9-ad83-d016fa2536f9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:21.247469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6df6a-f22b-46f9-ad83-d016fa2536f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:21.247752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6df6a-f22b-46f9-ad83-d016fa2536f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:21.248611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6df6a-f22b-46f9-ad83-d016fa2536f9 None admin] RBAC: Adding query filter params (role.id=0b98291295724747acd01dd0d2306e62, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51, user.id=264b2fc6044940aba215d38e72c6e93a) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:21.248935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6df6a-f22b-46f9-ad83-d016fa2536f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:21.250430 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6df6a-f22b-46f9-ad83-d016fa2536f9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:21.266789 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 78/151] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:21 2022] GET /identity/v3/role_assignments?user.id=264b2fc6044940aba215d38e72c6e93a&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=0b98291295724747acd01dd0d2306e62 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:21.306413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4914b0-0a84-418b-8123-6795af737a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:21.307253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4914b0-0a84-418b-8123-6795af737a2b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:21.307545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4914b0-0a84-418b-8123-6795af737a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:21.307820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4914b0-0a84-418b-8123-6795af737a2b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:21.330354 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4914b0-0a84-418b-8123-6795af737a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-1e4914b0-0a84-418b-8123-6795af737a2b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:21.339244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:21.340224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:21.341001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 74/152] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:21.346672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6df6a-f22b-46f9-ad83-d016fa2536f9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:21.478256 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6df6a-f22b-46f9-ad83-d016fa2536f9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:21.521652 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee004215-8637-4af3-a488-10770e95a4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:21.522381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee004215-8637-4af3-a488-10770e95a4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:21.522670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee004215-8637-4af3-a488-10770e95a4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:21.522952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee004215-8637-4af3-a488-10770e95a4e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:21.523732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee004215-8637-4af3-a488-10770e95a4e5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:21.524043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee004215-8637-4af3-a488-10770e95a4e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:21.525468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee004215-8637-4af3-a488-10770e95a4e5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:21.535039 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 79/153] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:21.541178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4914b0-0a84-418b-8123-6795af737a2b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:21.600043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:21.662293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4914b0-0a84-418b-8123-6795af737a2b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:21.698649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e1e47-e9b0-4f7f-9e61-f13fef141bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:21.699535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e1e47-e9b0-4f7f-9e61-f13fef141bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:21.699815 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e1e47-e9b0-4f7f-9e61-f13fef141bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:21.700086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e1e47-e9b0-4f7f-9e61-f13fef141bb2 None admin] PATH_INFO: `/v3/users/0f69336f9fdb4a51b0f910df3f54a336` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:21.701710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8e1e47-e9b0-4f7f-9e61-f13fef141bb2 None admin] RBAC: Authorizing `identity:get_user(user_id=0f69336f9fdb4a51b0f910df3f54a336)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:21.703210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8e1e47-e9b0-4f7f-9e61-f13fef141bb2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:21.704853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 75/154] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:21 2022] GET /identity/v3/users/0f69336f9fdb4a51b0f910df3f54a336 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:21.711195 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee004215-8637-4af3-a488-10770e95a4e5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:21.740231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:21.801281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563c36-d4f1-4413-88fe-12f285ae9da8 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:21.801785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563c36-d4f1-4413-88fe-12f285ae9da8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:21.801979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563c36-d4f1-4413-88fe-12f285ae9da8 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:21.803450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 76/155] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:21.808843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fff6e0-e223-412e-aa2a-a0be76622258 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:21.809132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fff6e0-e223-412e-aa2a-a0be76622258 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:21.809411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fff6e0-e223-412e-aa2a-a0be76622258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:21.839307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-09fff6e0-e223-412e-aa2a-a0be76622258 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:21.841067 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-09fff6e0-e223-412e-aa2a-a0be76622258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:21.844522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee004215-8637-4af3-a488-10770e95a4e5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:21.893478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956c797-d9ac-42b3-9b46-002608498e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:21.894953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956c797-d9ac-42b3-9b46-002608498e88 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:21.895766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956c797-d9ac-42b3-9b46-002608498e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:21.897291 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956c797-d9ac-42b3-9b46-002608498e88 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:21.898909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956c797-d9ac-42b3-9b46-002608498e88 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:21.900623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956c797-d9ac-42b3-9b46-002608498e88 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:21.903220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 80/156] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:21 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:21.909016 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956c797-d9ac-42b3-9b46-002608498e88 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:21.953518 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 77/157] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:22.023492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:22.032754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fff6e0-e223-412e-aa2a-a0be76622258 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:22.039925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956c797-d9ac-42b3-9b46-002608498e88 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:22.132646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:22.134464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118c9cb-3967-42e1-8383-969a89346fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:22.135654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c9cb-3967-42e1-8383-969a89346fd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:22.136119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c9cb-3967-42e1-8383-969a89346fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:22.136574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118c9cb-3967-42e1-8383-969a89346fd2 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/users/0f69336f9fdb4a51b0f910df3f54a336/roles/67473bdffb674c6b8d0afd413642eff6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:22.140285 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118c9cb-3967-42e1-8383-969a89346fd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=795c9a5eff6345f6ae2de8e24f361e51, user_id=0f69336f9fdb4a51b0f910df3f54a336, role_id=67473bdffb674c6b8d0afd413642eff6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:22.141991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118c9cb-3967-42e1-8383-969a89346fd2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:22.184147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 81/158] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:01:21 2022] PUT /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/users/0f69336f9fdb4a51b0f910df3f54a336/roles/67473bdffb674c6b8d0afd413642eff6 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:22.195666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fff6e0-e223-412e-aa2a-a0be76622258 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:22.235692 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df13fc2-63d7-4628-9176-892f08ed36a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:22.236508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df13fc2-63d7-4628-9176-892f08ed36a9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:22.236792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df13fc2-63d7-4628-9176-892f08ed36a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:22.237072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df13fc2-63d7-4628-9176-892f08ed36a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:22.249714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df13fc2-63d7-4628-9176-892f08ed36a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-9df13fc2-63d7-4628-9176-892f08ed36a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:22.259769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:22.260817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:22.261613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 78/159] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:01:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:22.267028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118c9cb-3967-42e1-8383-969a89346fd2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:22.387050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118c9cb-3967-42e1-8383-969a89346fd2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:22.429473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d021a-ecb5-4294-bea7-12140cbaba78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:22.430265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d021a-ecb5-4294-bea7-12140cbaba78 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:22.430554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d021a-ecb5-4294-bea7-12140cbaba78 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:22.430834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d021a-ecb5-4294-bea7-12140cbaba78 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:22.431967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d021a-ecb5-4294-bea7-12140cbaba78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:22.432292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d021a-ecb5-4294-bea7-12140cbaba78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:22.433777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d021a-ecb5-4294-bea7-12140cbaba78 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:22.448375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 82/160] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:22.453863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df13fc2-63d7-4628-9176-892f08ed36a9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:22.480847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fd148-5b77-483d-b2de-3c2bfd44718e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:22.481167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fd148-5b77-483d-b2de-3c2bfd44718e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:22.481403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fd148-5b77-483d-b2de-3c2bfd44718e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:22.482869 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 83/161] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:22.488272 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8890620-3ba0-45cf-8664-d859133fe44a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:22.488624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8890620-3ba0-45cf-8664-d859133fe44a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:22.488863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8890620-3ba0-45cf-8664-d859133fe44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:22.515778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f8890620-3ba0-45cf-8664-d859133fe44a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:22.516983 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f8890620-3ba0-45cf-8664-d859133fe44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:22.527096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:22.590616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 84/162] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:22.614924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8890620-3ba0-45cf-8664-d859133fe44a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:22.637923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df13fc2-63d7-4628-9176-892f08ed36a9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:22.655998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:22.720461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8890620-3ba0-45cf-8664-d859133fe44a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:22.734636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94043487-415d-4dba-959e-6cd3134fcfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:22.735547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94043487-415d-4dba-959e-6cd3134fcfb2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:22.735837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94043487-415d-4dba-959e-6cd3134fcfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:22.736116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94043487-415d-4dba-959e-6cd3134fcfb2 None admin] PATH_INFO: `/v3/users/78c2f3799fc8459a9a6e05f14a552498` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:22.737381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94043487-415d-4dba-959e-6cd3134fcfb2 None admin] RBAC: Authorizing `identity:get_user(user_id=78c2f3799fc8459a9a6e05f14a552498)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:22.738930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94043487-415d-4dba-959e-6cd3134fcfb2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:22.740713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 79/163] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:22 2022] GET /identity/v3/users/78c2f3799fc8459a9a6e05f14a552498 => generated 247 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:22.746418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94043487-415d-4dba-959e-6cd3134fcfb2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:22.755888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483f36-8b1d-4843-989b-3d25df0f1a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:22.756698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483f36-8b1d-4843-989b-3d25df0f1a46 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:22.756981 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483f36-8b1d-4843-989b-3d25df0f1a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:22.757256 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483f36-8b1d-4843-989b-3d25df0f1a46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:22.768460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb483f36-8b1d-4843-989b-3d25df0f1a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-bb483f36-8b1d-4843-989b-3d25df0f1a46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:22.776632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:22.777531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:22.778363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 85/164] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:22.784090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483f36-8b1d-4843-989b-3d25df0f1a46 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:22.788729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:22.913970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483f36-8b1d-4843-989b-3d25df0f1a46 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:22.957689 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348f320-bfad-465a-a038-09ad740b95f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:22.959317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f320-bfad-465a-a038-09ad740b95f4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:22.959958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f320-bfad-465a-a038-09ad740b95f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:22.960445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f320-bfad-465a-a038-09ad740b95f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:22.961426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348f320-bfad-465a-a038-09ad740b95f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:22.961954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348f320-bfad-465a-a038-09ad740b95f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:22.963863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348f320-bfad-465a-a038-09ad740b95f4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:22.973993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 86/165] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:22.979607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348f320-bfad-465a-a038-09ad740b95f4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:23.020676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94043487-415d-4dba-959e-6cd3134fcfb2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:23.086291 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:23.117960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90e726-ca14-4d5b-b701-09bd0b4137a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:23.121584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e726-ca14-4d5b-b701-09bd0b4137a9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:23.122107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e726-ca14-4d5b-b701-09bd0b4137a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:23.122614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90e726-ca14-4d5b-b701-09bd0b4137a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:23.148341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348f320-bfad-465a-a038-09ad740b95f4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:23.188174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:23.214958 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90e726-ca14-4d5b-b701-09bd0b4137a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:23.216815 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90e726-ca14-4d5b-b701-09bd0b4137a9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:23.222506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735062ae-3db8-42fc-b3ec-4981e40e3e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:23.223455 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735062ae-3db8-42fc-b3ec-4981e40e3e4e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:23.223745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735062ae-3db8-42fc-b3ec-4981e40e3e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:23.224016 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735062ae-3db8-42fc-b3ec-4981e40e3e4e None admin] PATH_INFO: `/v3/users/264b2fc6044940aba215d38e72c6e93a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:23.233543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735062ae-3db8-42fc-b3ec-4981e40e3e4e None admin] RBAC: Authorizing `identity:get_user(user_id=264b2fc6044940aba215d38e72c6e93a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:23.233543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735062ae-3db8-42fc-b3ec-4981e40e3e4e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:23.233970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 87/166] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:22 2022] GET /identity/v3/users/264b2fc6044940aba215d38e72c6e93a => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:23.239500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735062ae-3db8-42fc-b3ec-4981e40e3e4e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:23.376643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735062ae-3db8-42fc-b3ec-4981e40e3e4e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:23.412095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:23.413512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21029828-efca-47fb-8cc1-053b040a7ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:23.414144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21029828-efca-47fb-8cc1-053b040a7ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:23.414435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21029828-efca-47fb-8cc1-053b040a7ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:23.414718 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21029828-efca-47fb-8cc1-053b040a7ae8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:23.486942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21029828-efca-47fb-8cc1-053b040a7ae8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:23.489061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21029828-efca-47fb-8cc1-053b040a7ae8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:23.504681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 80/167] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:01:23.506125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7068473-0f48-46d3-9c64-afefadcafc58 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:23.506418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7068473-0f48-46d3-9c64-afefadcafc58 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:23.506697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7068473-0f48-46d3-9c64-afefadcafc58 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:23.508230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 81/168] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:23.512921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7068473-0f48-46d3-9c64-afefadcafc58 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:23.592216 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 88/169] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:01:23.593530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48efffa-fa61-4695-b4b5-b91b459139ab None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:23.598007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48efffa-fa61-4695-b4b5-b91b459139ab None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:23.598400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48efffa-fa61-4695-b4b5-b91b459139ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:23.622866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e48efffa-fa61-4695-b4b5-b91b459139ab None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:23.624295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e48efffa-fa61-4695-b4b5-b91b459139ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:23.652970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:23.677571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:23.704985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7068473-0f48-46d3-9c64-afefadcafc58 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:23.752496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 89/170] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:23.753641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48efffa-fa61-4695-b4b5-b91b459139ab None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:23.763160 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eefe72-4afa-4e84-969f-ae2583372b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:23.764193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eefe72-4afa-4e84-969f-ae2583372b3f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:23.764576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eefe72-4afa-4e84-969f-ae2583372b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:23.764950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eefe72-4afa-4e84-969f-ae2583372b3f None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:23.766367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eefe72-4afa-4e84-969f-ae2583372b3f None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:23.770474 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eefe72-4afa-4e84-969f-ae2583372b3f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:23.770850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 82/171] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:23 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:23.778730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eefe72-4afa-4e84-969f-ae2583372b3f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:23.878809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48efffa-fa61-4695-b4b5-b91b459139ab None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:23.916171 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d3ad8-14fe-4f41-b3c5-26316b487f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:23.917047 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3ad8-14fe-4f41-b3c5-26316b487f44 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:23.917332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3ad8-14fe-4f41-b3c5-26316b487f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:23.917723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3ad8-14fe-4f41-b3c5-26316b487f44 None admin] PATH_INFO: `/v3/projects/362499a7e2bf4afb8ebf317978db3c52` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:23.919047 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d3ad8-14fe-4f41-b3c5-26316b487f44 None admin] RBAC: Authorizing `identity:get_project(project_id=362499a7e2bf4afb8ebf317978db3c52)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:23.920551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d3ad8-14fe-4f41-b3c5-26316b487f44 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:23.922702 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 90/172] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:23 2022] GET /identity/v3/projects/362499a7e2bf4afb8ebf317978db3c52 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:23.923349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d3ad8-14fe-4f41-b3c5-26316b487f44 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:23.927401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eefe72-4afa-4e84-969f-ae2583372b3f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:23.961477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:23.970618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297112c-f0f3-4a72-833c-125603ad5a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:23.971296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297112c-f0f3-4a72-833c-125603ad5a28 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:23.971589 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297112c-f0f3-4a72-833c-125603ad5a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:23.971869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297112c-f0f3-4a72-833c-125603ad5a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:23.972723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297112c-f0f3-4a72-833c-125603ad5a28 None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51, user.id=78c2f3799fc8459a9a6e05f14a552498) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:23.973039 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297112c-f0f3-4a72-833c-125603ad5a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:23.974611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297112c-f0f3-4a72-833c-125603ad5a28 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:23.985131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 83/173] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:23 2022] GET /identity/v3/role_assignments?user.id=78c2f3799fc8459a9a6e05f14a552498&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=2302517080ac42578aa81c606dd0ff2f => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:23.985779 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297112c-f0f3-4a72-833c-125603ad5a28 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:24.000976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:24.049824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d3ad8-14fe-4f41-b3c5-26316b487f44 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:24.088163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ded0c7-b073-4a03-9d11-8cb697b2a631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:24.089285 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ded0c7-b073-4a03-9d11-8cb697b2a631 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:24.089713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ded0c7-b073-4a03-9d11-8cb697b2a631 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:24.090166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ded0c7-b073-4a03-9d11-8cb697b2a631 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:24.102977 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ded0c7-b073-4a03-9d11-8cb697b2a631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-16ded0c7-b073-4a03-9d11-8cb697b2a631 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:24.112076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:24.113025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:24.113921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:24.116677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 91/174] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:24.136252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ded0c7-b073-4a03-9d11-8cb697b2a631 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:24.145182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297112c-f0f3-4a72-833c-125603ad5a28 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:24.190083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f61e9e-2a59-441b-835f-66bc5ee87ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:24.190878 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f61e9e-2a59-441b-835f-66bc5ee87ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:24.191171 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f61e9e-2a59-441b-835f-66bc5ee87ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:24.191450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f61e9e-2a59-441b-835f-66bc5ee87ab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:24.192436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f61e9e-2a59-441b-835f-66bc5ee87ab9 None admin] RBAC: Adding query filter params (role.id=0b98291295724747acd01dd0d2306e62, scope.project.id=362499a7e2bf4afb8ebf317978db3c52, user.id=264b2fc6044940aba215d38e72c6e93a) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:24.192755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f61e9e-2a59-441b-835f-66bc5ee87ab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:24.194302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f61e9e-2a59-441b-835f-66bc5ee87ab9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:24.201119 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 84/175] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:23 2022] GET /identity/v3/role_assignments?user.id=264b2fc6044940aba215d38e72c6e93a&scope.project.id=362499a7e2bf4afb8ebf317978db3c52&role.id=0b98291295724747acd01dd0d2306e62 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:24.303479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:24.313496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ded0c7-b073-4a03-9d11-8cb697b2a631 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:24.359178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ec1fc-c156-415f-b2e8-78a2a0eb6e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:24.360206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ec1fc-c156-415f-b2e8-78a2a0eb6e42 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:24.360680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ec1fc-c156-415f-b2e8-78a2a0eb6e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:24.361147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ec1fc-c156-415f-b2e8-78a2a0eb6e42 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:24.362119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ec1fc-c156-415f-b2e8-78a2a0eb6e42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:24.362613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ec1fc-c156-415f-b2e8-78a2a0eb6e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:24.364578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ec1fc-c156-415f-b2e8-78a2a0eb6e42 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:24.378759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 92/176] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:24.384537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f61e9e-2a59-441b-835f-66bc5ee87ab9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:24.464014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:24.503208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f61e9e-2a59-441b-835f-66bc5ee87ab9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:24.544446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1893a7-b6fd-4e1c-ae0d-b9b62377cf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:24.545339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1893a7-b6fd-4e1c-ae0d-b9b62377cf8b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:24.545622 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1893a7-b6fd-4e1c-ae0d-b9b62377cf8b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:24.545910 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1893a7-b6fd-4e1c-ae0d-b9b62377cf8b None admin] PATH_INFO: `/v3/users/264b2fc6044940aba215d38e72c6e93a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:24.547152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1893a7-b6fd-4e1c-ae0d-b9b62377cf8b None admin] RBAC: Authorizing `identity:get_user(user_id=264b2fc6044940aba215d38e72c6e93a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:24.548605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1893a7-b6fd-4e1c-ae0d-b9b62377cf8b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:24.550585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 85/177] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:24 2022] GET /identity/v3/users/264b2fc6044940aba215d38e72c6e93a => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:24.556447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ec1fc-c156-415f-b2e8-78a2a0eb6e42 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:24.743276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ec1fc-c156-415f-b2e8-78a2a0eb6e42 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:24.789389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1e775-1f27-4759-8a88-7c19abfd58f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:24.790406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1e775-1f27-4759-8a88-7c19abfd58f7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:24.790882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1e775-1f27-4759-8a88-7c19abfd58f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:24.791329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1e775-1f27-4759-8a88-7c19abfd58f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:24.820093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:24.871923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1e775-1f27-4759-8a88-7c19abfd58f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:24.873710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1e775-1f27-4759-8a88-7c19abfd58f7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:24.875502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a430e-df0a-4005-b774-c98d56f8936c None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:24.875955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a430e-df0a-4005-b774-c98d56f8936c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:24.876044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a430e-df0a-4005-b774-c98d56f8936c None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:24.878281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 86/178] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:24.883240 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a578f-163c-4990-bd3a-f5cf8c477130 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:24.883554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a578f-163c-4990-bd3a-f5cf8c477130 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:24.884114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a578f-163c-4990-bd3a-f5cf8c477130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:24.919418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3a9a578f-163c-4990-bd3a-f5cf8c477130 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:24.922148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3a9a578f-163c-4990-bd3a-f5cf8c477130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:25.050965 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 93/179] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:25.056860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1e775-1f27-4759-8a88-7c19abfd58f7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:25.073348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 87/180] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:25.106818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a578f-163c-4990-bd3a-f5cf8c477130 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:25.146318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:25.195709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1e775-1f27-4759-8a88-7c19abfd58f7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:25.233437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4618b37-1e32-4b05-8ed8-413b18ab2776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:25.235085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4618b37-1e32-4b05-8ed8-413b18ab2776 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.235368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4618b37-1e32-4b05-8ed8-413b18ab2776 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.235641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4618b37-1e32-4b05-8ed8-413b18ab2776 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.237458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4618b37-1e32-4b05-8ed8-413b18ab2776 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:25.239026 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4618b37-1e32-4b05-8ed8-413b18ab2776 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:25.240802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 94/181] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:25 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:25.246710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4618b37-1e32-4b05-8ed8-413b18ab2776 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:25.286903 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:25.307647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a578f-163c-4990-bd3a-f5cf8c477130 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:25.349490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d525170-2ec6-4a1e-9da2-c32c4beae827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:25.350991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d525170-2ec6-4a1e-9da2-c32c4beae827 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.351490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d525170-2ec6-4a1e-9da2-c32c4beae827 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.352201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d525170-2ec6-4a1e-9da2-c32c4beae827 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.366596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d525170-2ec6-4a1e-9da2-c32c4beae827 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-8d525170-2ec6-4a1e-9da2-c32c4beae827 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:25.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:25.378921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:25.379967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 88/182] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:01:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:25.380722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26302-339b-4b47-8f19-49cb518cb981 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.381005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26302-339b-4b47-8f19-49cb518cb981 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.381279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26302-339b-4b47-8f19-49cb518cb981 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.383240 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 89/183] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:25.384539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e30427-da1d-445b-9175-406a9b18e817 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.385033 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e30427-da1d-445b-9175-406a9b18e817 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.385517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e30427-da1d-445b-9175-406a9b18e817 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.386978 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 90/184] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:25.387859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e30427-da1d-445b-9175-406a9b18e817 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:25.392707 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4618b37-1e32-4b05-8ed8-413b18ab2776 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:25.414783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:25.429259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37971d-ddba-4862-ae79-12a4678398f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:25.429942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37971d-ddba-4862-ae79-12a4678398f9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.430218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37971d-ddba-4862-ae79-12a4678398f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.430502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37971d-ddba-4862-ae79-12a4678398f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.431353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37971d-ddba-4862-ae79-12a4678398f9 None admin] RBAC: Adding query filter params (role.id=2eea147ab8b1426294a08106410f6d45, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51, user.id=264b2fc6044940aba215d38e72c6e93a) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:25.431664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37971d-ddba-4862-ae79-12a4678398f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:25.433239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37971d-ddba-4862-ae79-12a4678398f9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:25.439713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 95/185] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:25 2022] GET /identity/v3/role_assignments?user.id=264b2fc6044940aba215d38e72c6e93a&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=2eea147ab8b1426294a08106410f6d45 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:25.441086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab94fcb-00c1-4ffd-8131-66b691fb3bd5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.441533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab94fcb-00c1-4ffd-8131-66b691fb3bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.442091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab94fcb-00c1-4ffd-8131-66b691fb3bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.467936 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cab94fcb-00c1-4ffd-8131-66b691fb3bd5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:25.469445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cab94fcb-00c1-4ffd-8131-66b691fb3bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:25.516994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e30427-da1d-445b-9175-406a9b18e817 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:25.545272 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 96/186] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:25.546965 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150fc90-c889-447b-b983-2c62a968b841 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.547478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150fc90-c889-447b-b983-2c62a968b841 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.548035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150fc90-c889-447b-b983-2c62a968b841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.551140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:25.557413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b107ebc-ee0e-409a-8638-8b83755a0c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:25.558592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b107ebc-ee0e-409a-8638-8b83755a0c71 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.559174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b107ebc-ee0e-409a-8638-8b83755a0c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.559810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b107ebc-ee0e-409a-8638-8b83755a0c71 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.560719 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b107ebc-ee0e-409a-8638-8b83755a0c71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:25.561126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b107ebc-ee0e-409a-8638-8b83755a0c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:25.562821 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b107ebc-ee0e-409a-8638-8b83755a0c71 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:25.569562 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8150fc90-c889-447b-b983-2c62a968b841 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:25.571318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8150fc90-c889-447b-b983-2c62a968b841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:25.573304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 91/187] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:25.578819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b107ebc-ee0e-409a-8638-8b83755a0c71 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:25.646347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 97/188] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:25.647127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150fc90-c889-447b-b983-2c62a968b841 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:25.701267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b107ebc-ee0e-409a-8638-8b83755a0c71 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:25.704744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:25.742988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb5df5-c28a-44a1-b765-965183645374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:25.743873 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb5df5-c28a-44a1-b765-965183645374 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.744151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb5df5-c28a-44a1-b765-965183645374 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.744422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb5df5-c28a-44a1-b765-965183645374 None admin] PATH_INFO: `/v3/users/9e1438b660424e5db78809a8ea654dd0` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.758360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb5df5-c28a-44a1-b765-965183645374 None admin] RBAC: Authorizing `identity:get_user(user_id=9e1438b660424e5db78809a8ea654dd0)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:25.759821 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb5df5-c28a-44a1-b765-965183645374 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:25.761499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 92/189] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:25 2022] GET /identity/v3/users/9e1438b660424e5db78809a8ea654dd0 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:25.761499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841096a4-f328-477a-b2ac-32daffe0168f None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.761499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841096a4-f328-477a-b2ac-32daffe0168f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.761499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841096a4-f328-477a-b2ac-32daffe0168f None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.761499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 93/190] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:25.761499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841096a4-f328-477a-b2ac-32daffe0168f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:25.769178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150fc90-c889-447b-b983-2c62a968b841 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:25.803022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff1548-bee3-412a-b1bf-eb317e4585ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:25.803814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff1548-bee3-412a-b1bf-eb317e4585ec None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.804097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff1548-bee3-412a-b1bf-eb317e4585ec None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.804375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff1548-bee3-412a-b1bf-eb317e4585ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.805603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff1548-bee3-412a-b1bf-eb317e4585ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:25.807109 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff1548-bee3-412a-b1bf-eb317e4585ec None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:25.808743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 98/191] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:25.809494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff1548-bee3-412a-b1bf-eb317e4585ec None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:25.820821 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:25.871612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841096a4-f328-477a-b2ac-32daffe0168f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:25.905279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635c03d-2089-4aeb-93f8-d7c1c79ed5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:25.906219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635c03d-2089-4aeb-93f8-d7c1c79ed5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.906504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635c03d-2089-4aeb-93f8-d7c1c79ed5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.906781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635c03d-2089-4aeb-93f8-d7c1c79ed5f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.911694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff1548-bee3-412a-b1bf-eb317e4585ec None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:25.916902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635c03d-2089-4aeb-93f8-d7c1c79ed5f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:25.924460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-1635c03d-2089-4aeb-93f8-d7c1c79ed5f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:25.924939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:25.926231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:25.927505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:25.928725 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 94/192] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:25.930505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd3f2-a917-41c1-9a21-fa074c89e336 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.930790 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd3f2-a917-41c1-9a21-fa074c89e336 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.931065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd3f2-a917-41c1-9a21-fa074c89e336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:25.947339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-46bfd3f2-a917-41c1-9a21-fa074c89e336 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:25.947490 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e2e4b-025e-4d7c-8a44-779f9b349be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:25.948129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e2e4b-025e-4d7c-8a44-779f9b349be6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:25.948415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e2e4b-025e-4d7c-8a44-779f9b349be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:25.948558 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-46bfd3f2-a917-41c1-9a21-fa074c89e336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:25.948691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e2e4b-025e-4d7c-8a44-779f9b349be6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:26.015746 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 95/193] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:26.016628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd3f2-a917-41c1-9a21-fa074c89e336 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:26.017332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e2e4b-025e-4d7c-8a44-779f9b349be6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:26.018836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e2e4b-025e-4d7c-8a44-779f9b349be6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:26.066356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:26.126509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 99/194] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:26.127146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd3f2-a917-41c1-9a21-fa074c89e336 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:26.127463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e2e4b-025e-4d7c-8a44-779f9b349be6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:26.141636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:26.165165 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9834ad-9936-4921-ae68-9951d12cd99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:26.166081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9834ad-9936-4921-ae68-9951d12cd99d None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:26.166367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9834ad-9936-4921-ae68-9951d12cd99d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:26.166646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9834ad-9936-4921-ae68-9951d12cd99d None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:26.167382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9834ad-9936-4921-ae68-9951d12cd99d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:26.168895 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9834ad-9936-4921-ae68-9951d12cd99d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:26.193627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 96/195] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:01:26 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:26.193627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9834ad-9936-4921-ae68-9951d12cd99d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:26.235918 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e2e4b-025e-4d7c-8a44-779f9b349be6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:26.276059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce637a1-c34b-45d7-8774-895d5579311b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:26.276757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce637a1-c34b-45d7-8774-895d5579311b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:26.277044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce637a1-c34b-45d7-8774-895d5579311b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:26.277322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce637a1-c34b-45d7-8774-895d5579311b None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:26.278144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce637a1-c34b-45d7-8774-895d5579311b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:26.278459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce637a1-c34b-45d7-8774-895d5579311b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:26.279903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce637a1-c34b-45d7-8774-895d5579311b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:26.289105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 100/196] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:26.290835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce637a1-c34b-45d7-8774-895d5579311b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:26.312250 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:26.320267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9834ad-9936-4921-ae68-9951d12cd99d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:26.366234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e811-f6b8-4776-860b-e41d816a2565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:26.367044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e811-f6b8-4776-860b-e41d816a2565 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:26.367309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e811-f6b8-4776-860b-e41d816a2565 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:26.367580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0e811-f6b8-4776-860b-e41d816a2565 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:26.379291 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:26.382635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0e811-f6b8-4776-860b-e41d816a2565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-cbc0e811-f6b8-4776-860b-e41d816a2565 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:26.390588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:26.391478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:26.392299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 97/197] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:26.393611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e811-f6b8-4776-860b-e41d816a2565 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:26.415286 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce637a1-c34b-45d7-8774-895d5579311b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:26.457999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684e2bf-5780-4175-bde0-5504e33ee9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:26.458889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684e2bf-5780-4175-bde0-5504e33ee9cf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:26.459179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684e2bf-5780-4175-bde0-5504e33ee9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:26.459459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684e2bf-5780-4175-bde0-5504e33ee9cf None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:26.462845 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684e2bf-5780-4175-bde0-5504e33ee9cf None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:26.464472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684e2bf-5780-4175-bde0-5504e33ee9cf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:26.466598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 101/198] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:26 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:26.467385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684e2bf-5780-4175-bde0-5504e33ee9cf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:26.513615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0e811-f6b8-4776-860b-e41d816a2565 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:26.560224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cb028-23b7-49ad-a9ff-cf5c3c9b3172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:26.561470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb028-23b7-49ad-a9ff-cf5c3c9b3172 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:26.562310 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb028-23b7-49ad-a9ff-cf5c3c9b3172 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:26.562778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb028-23b7-49ad-a9ff-cf5c3c9b3172 None admin] PATH_INFO: `/v3/users/9e1438b660424e5db78809a8ea654dd0` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:26.565027 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cb028-23b7-49ad-a9ff-cf5c3c9b3172 None admin] RBAC: Authorizing `identity:get_user(user_id=9e1438b660424e5db78809a8ea654dd0)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:26.568063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cb028-23b7-49ad-a9ff-cf5c3c9b3172 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:26.570112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 98/199] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:26 2022] GET /identity/v3/users/9e1438b660424e5db78809a8ea654dd0 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:26.571124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cb028-23b7-49ad-a9ff-cf5c3c9b3172 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:26.602443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:26.610527 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684e2bf-5780-4175-bde0-5504e33ee9cf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:26.658902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcea90d-7bda-4f0e-ad99-6ec0bc0b7a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:26.659590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcea90d-7bda-4f0e-ad99-6ec0bc0b7a53 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:26.659877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcea90d-7bda-4f0e-ad99-6ec0bc0b7a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:26.660739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcea90d-7bda-4f0e-ad99-6ec0bc0b7a53 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:26.661517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcea90d-7bda-4f0e-ad99-6ec0bc0b7a53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:26.661831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcea90d-7bda-4f0e-ad99-6ec0bc0b7a53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:26.663720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcea90d-7bda-4f0e-ad99-6ec0bc0b7a53 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:26.675675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 102/200] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:26.676833 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcea90d-7bda-4f0e-ad99-6ec0bc0b7a53 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:26.682129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:26.688749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cb028-23b7-49ad-a9ff-cf5c3c9b3172 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:26.735549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd4a0c-785e-439d-960d-f88425961d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:26.736219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd4a0c-785e-439d-960d-f88425961d88 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:26.736504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd4a0c-785e-439d-960d-f88425961d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:26.736782 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd4a0c-785e-439d-960d-f88425961d88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:26.737672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd4a0c-785e-439d-960d-f88425961d88 None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=7e689a8addd941f8a2efd52508e178b7, user.id=9e1438b660424e5db78809a8ea654dd0) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:26.738499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd4a0c-785e-439d-960d-f88425961d88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:26.740679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd4a0c-785e-439d-960d-f88425961d88 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:26.748781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 99/201] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:26 2022] GET /identity/v3/role_assignments?user.id=9e1438b660424e5db78809a8ea654dd0&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=2302517080ac42578aa81c606dd0ff2f => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:26.749602 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd4a0c-785e-439d-960d-f88425961d88 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:26.858331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:26.873329 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd4a0c-785e-439d-960d-f88425961d88 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:26.912221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcea90d-7bda-4f0e-ad99-6ec0bc0b7a53 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:26.913378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fc775-ea82-4d4f-983f-8b9cbce8efc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:26.914415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fc775-ea82-4d4f-983f-8b9cbce8efc8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:26.915138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fc775-ea82-4d4f-983f-8b9cbce8efc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:26.915528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fc775-ea82-4d4f-983f-8b9cbce8efc8 None admin] PATH_INFO: `/v3/users/0f69336f9fdb4a51b0f910df3f54a336` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:26.917977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fc775-ea82-4d4f-983f-8b9cbce8efc8 None admin] RBAC: Authorizing `identity:get_user(user_id=0f69336f9fdb4a51b0f910df3f54a336)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:26.919582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fc775-ea82-4d4f-983f-8b9cbce8efc8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:26.921626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 100/202] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:26 2022] GET /identity/v3/users/0f69336f9fdb4a51b0f910df3f54a336 => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:26.930007 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fc775-ea82-4d4f-983f-8b9cbce8efc8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:26.989505 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7344068-cdc4-467a-a602-2325eae3b284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:26.992711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7344068-cdc4-467a-a602-2325eae3b284 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:26.993010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7344068-cdc4-467a-a602-2325eae3b284 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:26.993294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7344068-cdc4-467a-a602-2325eae3b284 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:27.095404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fc775-ea82-4d4f-983f-8b9cbce8efc8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:27.133918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a8975-5b46-4866-b739-9bd993331ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:27.137907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a8975-5b46-4866-b739-9bd993331ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:27.137907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a8975-5b46-4866-b739-9bd993331ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:27.137907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a8975-5b46-4866-b739-9bd993331ca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:27.163526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:27.171168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7344068-cdc4-467a-a602-2325eae3b284 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:27.172668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7344068-cdc4-467a-a602-2325eae3b284 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:27.179784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:27.210354 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a8975-5b46-4866-b739-9bd993331ca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:27.212032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a8975-5b46-4866-b739-9bd993331ca6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:27.321688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 101/203] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:01:27.343376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a8975-5b46-4866-b739-9bd993331ca6 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:27.402855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 103/204] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:27.409131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7344068-cdc4-467a-a602-2325eae3b284 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:27.454862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:27.529295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7344068-cdc4-467a-a602-2325eae3b284 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:27.532626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:27.556703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a8975-5b46-4866-b739-9bd993331ca6 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:27.570867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3075d8-44f9-4a68-be12-252b0cd2be2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:27.571748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3075d8-44f9-4a68-be12-252b0cd2be2c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:27.572032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3075d8-44f9-4a68-be12-252b0cd2be2c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:27.572308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3075d8-44f9-4a68-be12-252b0cd2be2c None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:27.577926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3075d8-44f9-4a68-be12-252b0cd2be2c None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:27.577926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3075d8-44f9-4a68-be12-252b0cd2be2c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:27.577926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 104/205] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:27 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:27.584093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3075d8-44f9-4a68-be12-252b0cd2be2c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:27.662674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362d959-a8cc-490c-81b9-8d3adf7336c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:27.663924 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362d959-a8cc-490c-81b9-8d3adf7336c8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:27.664402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362d959-a8cc-490c-81b9-8d3adf7336c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:27.664863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362d959-a8cc-490c-81b9-8d3adf7336c8 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:27.666347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362d959-a8cc-490c-81b9-8d3adf7336c8 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:27.667991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362d959-a8cc-490c-81b9-8d3adf7336c8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:27.669913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 102/206] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:27 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:27.678840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362d959-a8cc-490c-81b9-8d3adf7336c8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:27.700934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3075d8-44f9-4a68-be12-252b0cd2be2c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:27.746310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3393f-7e3b-42a4-bec5-8aae455eb795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:27.749085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3393f-7e3b-42a4-bec5-8aae455eb795 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:27.749563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3393f-7e3b-42a4-bec5-8aae455eb795 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:27.750740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3393f-7e3b-42a4-bec5-8aae455eb795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:27.752087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3393f-7e3b-42a4-bec5-8aae455eb795 None admin] RBAC: Adding query filter params (role.id=2eea147ab8b1426294a08106410f6d45, scope.project.id=7e689a8addd941f8a2efd52508e178b7, user.id=9e1438b660424e5db78809a8ea654dd0) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:27.757130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3393f-7e3b-42a4-bec5-8aae455eb795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:27.758852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3393f-7e3b-42a4-bec5-8aae455eb795 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:27.767694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 105/207] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:27 2022] GET /identity/v3/role_assignments?user.id=9e1438b660424e5db78809a8ea654dd0&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=2eea147ab8b1426294a08106410f6d45 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:27.873014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362d959-a8cc-490c-81b9-8d3adf7336c8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:27.909859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:27.925368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63697ce-23bf-44b7-a7c6-0df765b234ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:27.927658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63697ce-23bf-44b7-a7c6-0df765b234ba None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:27.928155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63697ce-23bf-44b7-a7c6-0df765b234ba None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:27.928629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63697ce-23bf-44b7-a7c6-0df765b234ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:27.929697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63697ce-23bf-44b7-a7c6-0df765b234ba None admin] RBAC: Adding query filter params (role.id=67473bdffb674c6b8d0afd413642eff6, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51, user.id=0f69336f9fdb4a51b0f910df3f54a336) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:27.930293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63697ce-23bf-44b7-a7c6-0df765b234ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:27.931961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63697ce-23bf-44b7-a7c6-0df765b234ba None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:27.943313 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 103/208] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:27 2022] GET /identity/v3/role_assignments?user.id=0f69336f9fdb4a51b0f910df3f54a336&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=67473bdffb674c6b8d0afd413642eff6 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:28.538917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79692b0d-1ccc-4e71-8538-89d03042717c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:28.539112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79692b0d-1ccc-4e71-8538-89d03042717c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:28.539395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79692b0d-1ccc-4e71-8538-89d03042717c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:28.540957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 106/209] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:28.546467 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7b6d-9c28-48dd-83a2-221cb8c3c2cc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:28.546985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7b6d-9c28-48dd-83a2-221cb8c3c2cc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:28.547499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7b6d-9c28-48dd-83a2-221cb8c3c2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:28.564927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e22b7b6d-9c28-48dd-83a2-221cb8c3c2cc None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:28.566495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e22b7b6d-9c28-48dd-83a2-221cb8c3c2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:28.630746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b67abc-cf12-41fa-a8fe-7bd22942f179 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:28.631053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b67abc-cf12-41fa-a8fe-7bd22942f179 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:28.631313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b67abc-cf12-41fa-a8fe-7bd22942f179 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:28.632952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 107/210] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:28.638202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115baaa3-912e-4423-af78-27edcde78662 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:28.638498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115baaa3-912e-4423-af78-27edcde78662 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:28.638781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115baaa3-912e-4423-af78-27edcde78662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:28.639820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 104/211] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:28.662554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-115baaa3-912e-4423-af78-27edcde78662 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:28.663762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-115baaa3-912e-4423-af78-27edcde78662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:28.667252 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b7b6d-9c28-48dd-83a2-221cb8c3c2cc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:28.696121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:28.755435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 108/212] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:28.781988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b7b6d-9c28-48dd-83a2-221cb8c3c2cc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:28.782163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115baaa3-912e-4423-af78-27edcde78662 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:28.803688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:28.829392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0de787-1496-4288-b10d-d3c7b82180da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:28.830236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0de787-1496-4288-b10d-d3c7b82180da None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:28.830579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0de787-1496-4288-b10d-d3c7b82180da None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:28.830903 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0de787-1496-4288-b10d-d3c7b82180da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:28.832093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0de787-1496-4288-b10d-d3c7b82180da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:28.833850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0de787-1496-4288-b10d-d3c7b82180da None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:28.836114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 105/213] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:28.841471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0de787-1496-4288-b10d-d3c7b82180da None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:28.899643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115baaa3-912e-4423-af78-27edcde78662 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:28.935472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f5a50-aeec-4465-879a-509ee4cae227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:28.936642 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f5a50-aeec-4465-879a-509ee4cae227 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:28.937124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f5a50-aeec-4465-879a-509ee4cae227 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:28.937627 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f5a50-aeec-4465-879a-509ee4cae227 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:28.947503 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0de787-1496-4288-b10d-d3c7b82180da None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:28.949710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f5a50-aeec-4465-879a-509ee4cae227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-480f5a50-aeec-4465-879a-509ee4cae227 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:28.957930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:28.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:28.959676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:28.962113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 109/214] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:28.967958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f5a50-aeec-4465-879a-509ee4cae227 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:28.988371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5462ccb-533b-4e16-8d02-1c1221c85f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:28.989090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462ccb-533b-4e16-8d02-1c1221c85f82 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:28.989375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462ccb-533b-4e16-8d02-1c1221c85f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:28.989654 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462ccb-533b-4e16-8d02-1c1221c85f82 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:28.990535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5462ccb-533b-4e16-8d02-1c1221c85f82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:28.991966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5462ccb-533b-4e16-8d02-1c1221c85f82 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:29.000072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:29.016070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 106/215] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:01:28 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:29.017216 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8009-a9b8-40b3-8140-e85e9195a461 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:29.017508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8009-a9b8-40b3-8140-e85e9195a461 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:29.017786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8009-a9b8-40b3-8140-e85e9195a461 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:29.019251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 107/216] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:29.025188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d6893-0097-4c83-a53f-9f3acfcb13ea None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:29.025491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d6893-0097-4c83-a53f-9f3acfcb13ea None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:29.025691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d6893-0097-4c83-a53f-9f3acfcb13ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:29.045935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-961d6893-0097-4c83-a53f-9f3acfcb13ea None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:29.047157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-961d6893-0097-4c83-a53f-9f3acfcb13ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:29.099186 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:29.121013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f5a50-aeec-4465-879a-509ee4cae227 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:29.150579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 108/217] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:29.161596 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a304cff-edda-4dbc-bf8b-7860fd773f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:29.162403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a304cff-edda-4dbc-bf8b-7860fd773f2d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:29.162740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a304cff-edda-4dbc-bf8b-7860fd773f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:29.163072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a304cff-edda-4dbc-bf8b-7860fd773f2d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:29.163914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a304cff-edda-4dbc-bf8b-7860fd773f2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:29.164279 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a304cff-edda-4dbc-bf8b-7860fd773f2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:29.165752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a304cff-edda-4dbc-bf8b-7860fd773f2d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:29.176198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 110/218] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:29.177487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d6893-0097-4c83-a53f-9f3acfcb13ea None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:29.182978 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a304cff-edda-4dbc-bf8b-7860fd773f2d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:29.301689 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:29.310798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d6893-0097-4c83-a53f-9f3acfcb13ea None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:29.318073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a304cff-edda-4dbc-bf8b-7860fd773f2d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:29.351121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac3c1c-49dd-4b2f-b85f-ef00dca4c064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:29.351855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac3c1c-49dd-4b2f-b85f-ef00dca4c064 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:29.352139 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac3c1c-49dd-4b2f-b85f-ef00dca4c064 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:29.352412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac3c1c-49dd-4b2f-b85f-ef00dca4c064 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:29.362126 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac77625-79ee-45fd-9b01-b9d42f1278dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:29.363614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac77625-79ee-45fd-9b01-b9d42f1278dc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:29.364120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac77625-79ee-45fd-9b01-b9d42f1278dc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:29.364551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac77625-79ee-45fd-9b01-b9d42f1278dc None admin] PATH_INFO: `/v3/users/1b64775d0c0f40869a3e56ebb647a5d3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:29.366134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac3c1c-49dd-4b2f-b85f-ef00dca4c064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-b6ac3c1c-49dd-4b2f-b85f-ef00dca4c064 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:29.373754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:29.374881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:29.375979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 109/219] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:01:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:29.384043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac3c1c-49dd-4b2f-b85f-ef00dca4c064 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:29.384415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac77625-79ee-45fd-9b01-b9d42f1278dc None admin] RBAC: Authorizing `identity:get_user(user_id=1b64775d0c0f40869a3e56ebb647a5d3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:29.386175 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac77625-79ee-45fd-9b01-b9d42f1278dc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:29.389219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 111/220] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:29 2022] GET /identity/v3/users/1b64775d0c0f40869a3e56ebb647a5d3 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:29.395636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac77625-79ee-45fd-9b01-b9d42f1278dc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:29.420627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:29.505302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac3c1c-49dd-4b2f-b85f-ef00dca4c064 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:29.541276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c6d51-417b-40d0-85da-91dd70424b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:29.542086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c6d51-417b-40d0-85da-91dd70424b07 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:29.542443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c6d51-417b-40d0-85da-91dd70424b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:29.542789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c6d51-417b-40d0-85da-91dd70424b07 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:29.543621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c6d51-417b-40d0-85da-91dd70424b07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:29.544000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c6d51-417b-40d0-85da-91dd70424b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:29.545569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c6d51-417b-40d0-85da-91dd70424b07 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:29.555308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac77625-79ee-45fd-9b01-b9d42f1278dc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:29.555669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 110/221] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:29.565632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c6d51-417b-40d0-85da-91dd70424b07 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:29.649766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b95be-c969-4eb8-b19c-a08b219fb6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:29.650386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b95be-c969-4eb8-b19c-a08b219fb6ca None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:29.650672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b95be-c969-4eb8-b19c-a08b219fb6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:29.650955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b95be-c969-4eb8-b19c-a08b219fb6ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:29.669083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c6d51-417b-40d0-85da-91dd70424b07 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:29.691097 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:29.699065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:29.705294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef35e9-bc8e-43a7-84f5-407a541f3a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:29.706253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef35e9-bc8e-43a7-84f5-407a541f3a08 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:29.706549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef35e9-bc8e-43a7-84f5-407a541f3a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:29.706828 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef35e9-bc8e-43a7-84f5-407a541f3a08 None admin] PATH_INFO: `/v3/users/9e1438b660424e5db78809a8ea654dd0` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:29.708409 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef35e9-bc8e-43a7-84f5-407a541f3a08 None admin] RBAC: Authorizing `identity:get_user(user_id=9e1438b660424e5db78809a8ea654dd0)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:29.710231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef35e9-bc8e-43a7-84f5-407a541f3a08 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:29.712338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 111/222] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:29 2022] GET /identity/v3/users/9e1438b660424e5db78809a8ea654dd0 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:29.717868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef35e9-bc8e-43a7-84f5-407a541f3a08 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:29.801005 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b95be-c969-4eb8-b19c-a08b219fb6ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:29.802571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b95be-c969-4eb8-b19c-a08b219fb6ca None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:29.900183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef35e9-bc8e-43a7-84f5-407a541f3a08 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:29.941831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 112/223] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:29.948081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b95be-c969-4eb8-b19c-a08b219fb6ca None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:29.969371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934fbff-9061-41bb-970a-c6b9b2cf0a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:29.970274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934fbff-9061-41bb-970a-c6b9b2cf0a8c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:29.970557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934fbff-9061-41bb-970a-c6b9b2cf0a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:29.970829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934fbff-9061-41bb-970a-c6b9b2cf0a8c None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:29.974700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934fbff-9061-41bb-970a-c6b9b2cf0a8c None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:29.976186 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934fbff-9061-41bb-970a-c6b9b2cf0a8c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:29.977927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 112/224] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:29 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:29.985720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934fbff-9061-41bb-970a-c6b9b2cf0a8c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:30.022279 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:30.078312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:30.091723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b95be-c969-4eb8-b19c-a08b219fb6ca None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:30.155458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2217f37-4a77-4cbc-b9de-fc8fc639c0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:30.156140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2217f37-4a77-4cbc-b9de-fc8fc639c0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:30.156431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2217f37-4a77-4cbc-b9de-fc8fc639c0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:30.156709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2217f37-4a77-4cbc-b9de-fc8fc639c0e2 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:30.162700 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2217f37-4a77-4cbc-b9de-fc8fc639c0e2 None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:30.164190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2217f37-4a77-4cbc-b9de-fc8fc639c0e2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:30.168219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 113/225] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:29 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:30.173848 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2217f37-4a77-4cbc-b9de-fc8fc639c0e2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:30.188984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934fbff-9061-41bb-970a-c6b9b2cf0a8c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:30.228569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed518c-c974-491f-b48d-593fb58b0a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:30.229475 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed518c-c974-491f-b48d-593fb58b0a8c None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:30.229762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed518c-c974-491f-b48d-593fb58b0a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:30.230073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed518c-c974-491f-b48d-593fb58b0a8c None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7/users/9e1438b660424e5db78809a8ea654dd0/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:30.233068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed518c-c974-491f-b48d-593fb58b0a8c None admin] RBAC: Authorizing `identity:create_grant(project_id=7e689a8addd941f8a2efd52508e178b7, user_id=9e1438b660424e5db78809a8ea654dd0, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:30.234571 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed518c-c974-491f-b48d-593fb58b0a8c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:30.248968 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 113/226] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:01:29 2022] PUT /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7/users/9e1438b660424e5db78809a8ea654dd0/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:30.250595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe77b98-00e4-42e5-b583-1f36cc09f8de None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:30.250892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe77b98-00e4-42e5-b583-1f36cc09f8de None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:30.251178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe77b98-00e4-42e5-b583-1f36cc09f8de None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:30.253045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 114/227] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:30.259475 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e2996-630d-479a-961b-47fe1d3663ab None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:30.259787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e2996-630d-479a-961b-47fe1d3663ab None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:30.260049 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e2996-630d-479a-961b-47fe1d3663ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:30.280176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-774e2996-630d-479a-961b-47fe1d3663ab None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:30.281608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-774e2996-630d-479a-961b-47fe1d3663ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:30.305463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2217f37-4a77-4cbc-b9de-fc8fc639c0e2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:30.321508 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:30.362057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 115/228] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:30.370713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69307c90-ab2b-4859-a40e-6a51e89fb332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:30.371407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69307c90-ab2b-4859-a40e-6a51e89fb332 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:30.371693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69307c90-ab2b-4859-a40e-6a51e89fb332 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:30.371967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69307c90-ab2b-4859-a40e-6a51e89fb332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:30.372827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69307c90-ab2b-4859-a40e-6a51e89fb332 None admin] RBAC: Adding query filter params (role.id=0b98291295724747acd01dd0d2306e62, scope.project.id=7e689a8addd941f8a2efd52508e178b7, user.id=1b64775d0c0f40869a3e56ebb647a5d3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:30.373138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69307c90-ab2b-4859-a40e-6a51e89fb332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:30.374626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69307c90-ab2b-4859-a40e-6a51e89fb332 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:30.381228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 114/229] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:30 2022] GET /identity/v3/role_assignments?user.id=1b64775d0c0f40869a3e56ebb647a5d3&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=0b98291295724747acd01dd0d2306e62 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:30.385027 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e2996-630d-479a-961b-47fe1d3663ab None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:30.410742 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:30.507852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e2996-630d-479a-961b-47fe1d3663ab None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:30.550514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa920c7-8fea-46d4-bcb7-b5d93efe5db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:30.551401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa920c7-8fea-46d4-bcb7-b5d93efe5db2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:30.551682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa920c7-8fea-46d4-bcb7-b5d93efe5db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:30.551954 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa920c7-8fea-46d4-bcb7-b5d93efe5db2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:30.564162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa920c7-8fea-46d4-bcb7-b5d93efe5db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-caa920c7-8fea-46d4-bcb7-b5d93efe5db2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:30.572566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:30.573567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:30.574404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 116/230] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:30.581501 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69307c90-ab2b-4859-a40e-6a51e89fb332 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:30.691544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69307c90-ab2b-4859-a40e-6a51e89fb332 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:30.726017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720234b8-34c4-43a5-aa3d-48f21bae3220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:30.727051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720234b8-34c4-43a5-aa3d-48f21bae3220 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:30.727535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720234b8-34c4-43a5-aa3d-48f21bae3220 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:30.728093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720234b8-34c4-43a5-aa3d-48f21bae3220 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:30.729044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720234b8-34c4-43a5-aa3d-48f21bae3220 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:30.729561 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720234b8-34c4-43a5-aa3d-48f21bae3220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:30.731179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720234b8-34c4-43a5-aa3d-48f21bae3220 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:30.740095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 115/231] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:30.745809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa920c7-8fea-46d4-bcb7-b5d93efe5db2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:30.877800 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:30.906143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa920c7-8fea-46d4-bcb7-b5d93efe5db2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:30.956484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c94564-d7ef-4de1-b544-0f286cae9935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:30.957282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c94564-d7ef-4de1-b544-0f286cae9935 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:30.957660 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c94564-d7ef-4de1-b544-0f286cae9935 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:30.957971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c94564-d7ef-4de1-b544-0f286cae9935 None admin] PATH_INFO: `/v3/users/9e1438b660424e5db78809a8ea654dd0` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:30.959205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c94564-d7ef-4de1-b544-0f286cae9935 None admin] RBAC: Authorizing `identity:get_user(user_id=9e1438b660424e5db78809a8ea654dd0)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:30.960691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c94564-d7ef-4de1-b544-0f286cae9935 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:30.962383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 117/232] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:30 2022] GET /identity/v3/users/9e1438b660424e5db78809a8ea654dd0 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:30.968124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720234b8-34c4-43a5-aa3d-48f21bae3220 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:31.024936 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:31.110014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720234b8-34c4-43a5-aa3d-48f21bae3220 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:31.175493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f40de0-41eb-4d39-a948-dfc4d384f459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:31.176714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f40de0-41eb-4d39-a948-dfc4d384f459 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:31.177003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f40de0-41eb-4d39-a948-dfc4d384f459 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:31.177287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f40de0-41eb-4d39-a948-dfc4d384f459 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:31.178543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f40de0-41eb-4d39-a948-dfc4d384f459 None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:31.180032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f40de0-41eb-4d39-a948-dfc4d384f459 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:31.183160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 116/233] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:30 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:31.188208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c94564-d7ef-4de1-b544-0f286cae9935 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:31.375641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c94564-d7ef-4de1-b544-0f286cae9935 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:31.430881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6eab45-82c6-42ed-8146-d319df5c52c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:31.431965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6eab45-82c6-42ed-8146-d319df5c52c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:31.432253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6eab45-82c6-42ed-8146-d319df5c52c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:31.432528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6eab45-82c6-42ed-8146-d319df5c52c5 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7/users/9e1438b660424e5db78809a8ea654dd0/roles/2eea147ab8b1426294a08106410f6d45` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:31.439721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6eab45-82c6-42ed-8146-d319df5c52c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e689a8addd941f8a2efd52508e178b7, user_id=9e1438b660424e5db78809a8ea654dd0, role_id=2eea147ab8b1426294a08106410f6d45)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:31.441201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6eab45-82c6-42ed-8146-d319df5c52c5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:31.472521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 118/234] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:01:31 2022] PUT /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7/users/9e1438b660424e5db78809a8ea654dd0/roles/2eea147ab8b1426294a08106410f6d45 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:31.707165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd261c-75df-47a1-ab1d-b197e8bf1615 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:31.707432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd261c-75df-47a1-ab1d-b197e8bf1615 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:31.707819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd261c-75df-47a1-ab1d-b197e8bf1615 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:31.709582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 117/235] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:31.715280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c9c10-d987-49a5-8eb3-03fd84ae0a78 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:31.715569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c9c10-d987-49a5-8eb3-03fd84ae0a78 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:31.715846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c9c10-d987-49a5-8eb3-03fd84ae0a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:31.727921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef868e-1369-4926-becc-a18abc99ad27 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:31.728214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef868e-1369-4926-becc-a18abc99ad27 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:31.728505 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef868e-1369-4926-becc-a18abc99ad27 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:31.730228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 118/236] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:31.738970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d72c9c10-d987-49a5-8eb3-03fd84ae0a78 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:31.740411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d72c9c10-d987-49a5-8eb3-03fd84ae0a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:31.741702 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035781bc-603c-4019-9fd3-43cfe82d127c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:31.742028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035781bc-603c-4019-9fd3-43cfe82d127c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:31.742305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035781bc-603c-4019-9fd3-43cfe82d127c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:31.743050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:31.766238 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-035781bc-603c-4019-9fd3-43cfe82d127c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:31.767426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-035781bc-603c-4019-9fd3-43cfe82d127c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:31.829020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 119/237] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:31.854510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c9c10-d987-49a5-8eb3-03fd84ae0a78 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:31.855399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 119/238] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:31.880344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035781bc-603c-4019-9fd3-43cfe82d127c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:31.920726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:31.962911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c9c10-d987-49a5-8eb3-03fd84ae0a78 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:31.985138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035781bc-603c-4019-9fd3-43cfe82d127c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:31.999028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095437c-c29d-4798-a023-113ac6bac47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:31.999844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095437c-c29d-4798-a023-113ac6bac47d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:32.000124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095437c-c29d-4798-a023-113ac6bac47d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:32.000450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095437c-c29d-4798-a023-113ac6bac47d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:32.002009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095437c-c29d-4798-a023-113ac6bac47d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:32.003484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095437c-c29d-4798-a023-113ac6bac47d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:32.005469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 120/239] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:32.011003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095437c-c29d-4798-a023-113ac6bac47d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:32.020874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd7de3-c1ef-4f56-879d-c41fc0c0f673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:32.021456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:32.021685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd7de3-c1ef-4f56-879d-c41fc0c0f673 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:32.022014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd7de3-c1ef-4f56-879d-c41fc0c0f673 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:32.022294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd7de3-c1ef-4f56-879d-c41fc0c0f673 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:32.034556 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd7de3-c1ef-4f56-879d-c41fc0c0f673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-51bd7de3-c1ef-4f56-879d-c41fc0c0f673 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:32.042582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:32.043495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:32.045692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 120/240] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:32.049364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd7de3-c1ef-4f56-879d-c41fc0c0f673 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:32.124756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095437c-c29d-4798-a023-113ac6bac47d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:32.164866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f301e-76b3-4e47-8e04-5e77f59ca147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:32.165694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f301e-76b3-4e47-8e04-5e77f59ca147 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:32.166014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f301e-76b3-4e47-8e04-5e77f59ca147 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:32.166294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f301e-76b3-4e47-8e04-5e77f59ca147 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:32.167023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f301e-76b3-4e47-8e04-5e77f59ca147 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:32.168484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f301e-76b3-4e47-8e04-5e77f59ca147 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:32.195529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 121/241] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:01:32 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:01:32.256600 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:32.260659 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd7de3-c1ef-4f56-879d-c41fc0c0f673 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:32.310653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a6d05-eaee-4eb0-9526-2f3fa952d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:32.311421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a6d05-eaee-4eb0-9526-2f3fa952d37d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:32.311708 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a6d05-eaee-4eb0-9526-2f3fa952d37d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:32.311985 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a6d05-eaee-4eb0-9526-2f3fa952d37d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:32.312811 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a6d05-eaee-4eb0-9526-2f3fa952d37d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:32.313123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a6d05-eaee-4eb0-9526-2f3fa952d37d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:32.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a6d05-eaee-4eb0-9526-2f3fa952d37d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:32.324370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 121/242] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:32.330178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f301e-76b3-4e47-8e04-5e77f59ca147 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:32.421841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:32.465585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f301e-76b3-4e47-8e04-5e77f59ca147 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:32.499849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd3ca8-8f38-458b-b663-c33e2b39acbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:32.500740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd3ca8-8f38-458b-b663-c33e2b39acbf None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:32.501024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd3ca8-8f38-458b-b663-c33e2b39acbf None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:32.501300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd3ca8-8f38-458b-b663-c33e2b39acbf None admin] PATH_INFO: `/v3/users/3a51b27be6924ac58c85b3f110cfbf7e` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:32.516021 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd3ca8-8f38-458b-b663-c33e2b39acbf None admin] RBAC: Authorizing `identity:get_user(user_id=3a51b27be6924ac58c85b3f110cfbf7e)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:32.517434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd3ca8-8f38-458b-b663-c33e2b39acbf None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:32.519205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 122/243] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:32 2022] GET /identity/v3/users/3a51b27be6924ac58c85b3f110cfbf7e => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:32.525206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a6d05-eaee-4eb0-9526-2f3fa952d37d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:32.655450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a6d05-eaee-4eb0-9526-2f3fa952d37d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:32.696482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0167a-1ab8-4999-9436-0912afa79d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:32.697141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0167a-1ab8-4999-9436-0912afa79d1b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:32.697423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0167a-1ab8-4999-9436-0912afa79d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:32.697697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0167a-1ab8-4999-9436-0912afa79d1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:32.704921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e511d-dadd-4326-ae4f-319c4d7ff3c3 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:32.705213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e511d-dadd-4326-ae4f-319c4d7ff3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:32.705489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e511d-dadd-4326-ae4f-319c4d7ff3c3 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:32.707684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 123/244] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:32.713257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6cf8-6a80-4d44-b0ea-5f9334833a7d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:32.713813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6cf8-6a80-4d44-b0ea-5f9334833a7d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:32.714326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6cf8-6a80-4d44-b0ea-5f9334833a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:32.734209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:32.735006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e61f6cf8-6a80-4d44-b0ea-5f9334833a7d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:32.736412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e61f6cf8-6a80-4d44-b0ea-5f9334833a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:32.787061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0167a-1ab8-4999-9436-0912afa79d1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:32.790347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0167a-1ab8-4999-9436-0912afa79d1b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:32.818468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 124/245] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 04 09:01:32.849304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6cf8-6a80-4d44-b0ea-5f9334833a7d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:32.907061 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:32.910113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 122/246] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:32.918657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0167a-1ab8-4999-9436-0912afa79d1b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:32.973867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6cf8-6a80-4d44-b0ea-5f9334833a7d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:32.994527 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:33.027943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b1805-3585-43c3-9d46-32315e52635f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:33.029093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b1805-3585-43c3-9d46-32315e52635f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:33.029572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b1805-3585-43c3-9d46-32315e52635f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:33.030139 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b1805-3585-43c3-9d46-32315e52635f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:33.043722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b1805-3585-43c3-9d46-32315e52635f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:33.050976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0167a-1ab8-4999-9436-0912afa79d1b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-856b1805-3585-43c3-9d46-32315e52635f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:33.052668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:33.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:33.054687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:33.061010 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 125/247] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:33.063659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b1805-3585-43c3-9d46-32315e52635f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:33.099838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bc984-7f39-45d2-a296-fcd60510a54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:33.100718 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bc984-7f39-45d2-a296-fcd60510a54a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:33.100996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bc984-7f39-45d2-a296-fcd60510a54a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:33.101267 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bc984-7f39-45d2-a296-fcd60510a54a None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:33.103885 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9bc984-7f39-45d2-a296-fcd60510a54a None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:33.106473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9bc984-7f39-45d2-a296-fcd60510a54a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:33.108220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 123/248] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:32 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:33.115089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bc984-7f39-45d2-a296-fcd60510a54a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:33.209101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b1805-3585-43c3-9d46-32315e52635f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:33.220686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:33.246030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e863f-a62d-4769-8537-82cbfa017734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:33.246712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e863f-a62d-4769-8537-82cbfa017734 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:33.247096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e863f-a62d-4769-8537-82cbfa017734 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:33.247375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e863f-a62d-4769-8537-82cbfa017734 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:33.248244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e863f-a62d-4769-8537-82cbfa017734 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:33.248551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e863f-a62d-4769-8537-82cbfa017734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:33.249986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e863f-a62d-4769-8537-82cbfa017734 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:33.259600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 126/249] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:33.260925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b1f0d-0abb-4c65-924a-57e327b4c726 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:33.264647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b1f0d-0abb-4c65-924a-57e327b4c726 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:33.265754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b1f0d-0abb-4c65-924a-57e327b4c726 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:33.267416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 127/250] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:33.269762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b1f0d-0abb-4c65-924a-57e327b4c726 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:33.310621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bc984-7f39-45d2-a296-fcd60510a54a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:33.358450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:33.371256 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94699b35-49c2-4fd7-b5e6-0a67aaed485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:33.372022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94699b35-49c2-4fd7-b5e6-0a67aaed485e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:33.372353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94699b35-49c2-4fd7-b5e6-0a67aaed485e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:33.372635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94699b35-49c2-4fd7-b5e6-0a67aaed485e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:33.373479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94699b35-49c2-4fd7-b5e6-0a67aaed485e None admin] RBAC: Adding query filter params (role.id=67473bdffb674c6b8d0afd413642eff6, scope.project.id=7e689a8addd941f8a2efd52508e178b7, user.id=3a51b27be6924ac58c85b3f110cfbf7e) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:33.373816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94699b35-49c2-4fd7-b5e6-0a67aaed485e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:33.375399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94699b35-49c2-4fd7-b5e6-0a67aaed485e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:33.386412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b1f0d-0abb-4c65-924a-57e327b4c726 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:33.396733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 124/251] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:33 2022] GET /identity/v3/role_assignments?user.id=3a51b27be6924ac58c85b3f110cfbf7e&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=67473bdffb674c6b8d0afd413642eff6 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:33.397808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa996b-c3af-45ab-b4b5-226b552fe7b3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:33.398129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa996b-c3af-45ab-b4b5-226b552fe7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:33.398411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa996b-c3af-45ab-b4b5-226b552fe7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:33.424500 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317c6c6-e8a7-427c-86b5-48db4276ca04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:33.425899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317c6c6-e8a7-427c-86b5-48db4276ca04 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:33.426784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317c6c6-e8a7-427c-86b5-48db4276ca04 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:33.427424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317c6c6-e8a7-427c-86b5-48db4276ca04 None admin] PATH_INFO: `/v3/users/9e1438b660424e5db78809a8ea654dd0` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:33.428955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317c6c6-e8a7-427c-86b5-48db4276ca04 None admin] RBAC: Authorizing `identity:get_user(user_id=9e1438b660424e5db78809a8ea654dd0)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:33.430803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317c6c6-e8a7-427c-86b5-48db4276ca04 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:33.433174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 128/252] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:33 2022] GET /identity/v3/users/9e1438b660424e5db78809a8ea654dd0 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:33.438471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-98fa996b-c3af-45ab-b4b5-226b552fe7b3 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:33.438641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317c6c6-e8a7-427c-86b5-48db4276ca04 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:33.439694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-98fa996b-c3af-45ab-b4b5-226b552fe7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:33.490254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:33.554519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317c6c6-e8a7-427c-86b5-48db4276ca04 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:33.594531 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756480aa-70b7-477b-909d-391adf34db30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:33.595835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756480aa-70b7-477b-909d-391adf34db30 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:33.596193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756480aa-70b7-477b-909d-391adf34db30 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:33.596551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756480aa-70b7-477b-909d-391adf34db30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:33.650710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 125/253] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:33.683713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756480aa-70b7-477b-909d-391adf34db30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:33.685331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756480aa-70b7-477b-909d-391adf34db30 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:33.740027 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:33.741300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa996b-c3af-45ab-b4b5-226b552fe7b3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:33.804799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 129/254] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:33.810607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756480aa-70b7-477b-909d-391adf34db30 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:33.914596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756480aa-70b7-477b-909d-391adf34db30 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:33.934470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa996b-c3af-45ab-b4b5-226b552fe7b3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:33.951838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdffa6-b8d4-49db-943e-52b5fd961d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:33.953198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdffa6-b8d4-49db-943e-52b5fd961d30 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:33.953675 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdffa6-b8d4-49db-943e-52b5fd961d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:33.954190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bdffa6-b8d4-49db-943e-52b5fd961d30 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:33.955699 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bdffa6-b8d4-49db-943e-52b5fd961d30 None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:33.957467 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bdffa6-b8d4-49db-943e-52b5fd961d30 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:33.958721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:33.961031 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 130/255] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:33 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:33.967086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdffa6-b8d4-49db-943e-52b5fd961d30 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:33.970914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ef935-77bf-4b84-a680-171046e6c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:33.971959 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ef935-77bf-4b84-a680-171046e6c327 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:33.972447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ef935-77bf-4b84-a680-171046e6c327 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:33.972907 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ef935-77bf-4b84-a680-171046e6c327 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:33.984395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ef935-77bf-4b84-a680-171046e6c327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:33.986209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1a9ef935-77bf-4b84-a680-171046e6c327 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:33.993131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:33.994103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:33.995392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:33.997941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 126/256] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:34.004112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ef935-77bf-4b84-a680-171046e6c327 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:34.117486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bdffa6-b8d4-49db-943e-52b5fd961d30 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:34.206571 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5b9d5-3fa2-4295-8330-4fa0c8cd5b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:34.207143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5b9d5-3fa2-4295-8330-4fa0c8cd5b78 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:34.207510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5b9d5-3fa2-4295-8330-4fa0c8cd5b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:34.207824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5b9d5-3fa2-4295-8330-4fa0c8cd5b78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:34.208666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5b9d5-3fa2-4295-8330-4fa0c8cd5b78 None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=7e689a8addd941f8a2efd52508e178b7, user.id=9e1438b660424e5db78809a8ea654dd0) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:34.208975 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5b9d5-3fa2-4295-8330-4fa0c8cd5b78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:34.210451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5b9d5-3fa2-4295-8330-4fa0c8cd5b78 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:34.225418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 131/257] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:33 2022] GET /identity/v3/role_assignments?user.id=9e1438b660424e5db78809a8ea654dd0&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=2302517080ac42578aa81c606dd0ff2f => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:34.247897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ef935-77bf-4b84-a680-171046e6c327 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:34.291423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb55f63-72f4-4470-b351-a2d0fc5df0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:34.292111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb55f63-72f4-4470-b351-a2d0fc5df0ef None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:34.292392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb55f63-72f4-4470-b351-a2d0fc5df0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:34.292666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb55f63-72f4-4470-b351-a2d0fc5df0ef None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:34.293446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb55f63-72f4-4470-b351-a2d0fc5df0ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:34.293750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb55f63-72f4-4470-b351-a2d0fc5df0ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:34.295234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb55f63-72f4-4470-b351-a2d0fc5df0ef None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:34.310320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 127/258] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:34.316359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5b9d5-3fa2-4295-8330-4fa0c8cd5b78 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:34.411946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:34.456323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5b9d5-3fa2-4295-8330-4fa0c8cd5b78 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:34.503352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acbbc2-cbfe-4490-b174-fa304c9868ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:34.504204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acbbc2-cbfe-4490-b174-fa304c9868ca None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:34.504489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acbbc2-cbfe-4490-b174-fa304c9868ca None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:34.504760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acbbc2-cbfe-4490-b174-fa304c9868ca None admin] PATH_INFO: `/v3/users/1b64775d0c0f40869a3e56ebb647a5d3` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:34.506150 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acbbc2-cbfe-4490-b174-fa304c9868ca None admin] RBAC: Authorizing `identity:get_user(user_id=1b64775d0c0f40869a3e56ebb647a5d3)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:34.507620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acbbc2-cbfe-4490-b174-fa304c9868ca None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:34.509556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 132/259] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:34 2022] GET /identity/v3/users/1b64775d0c0f40869a3e56ebb647a5d3 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:34.515007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb55f63-72f4-4470-b351-a2d0fc5df0ef None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:34.562435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:34.625892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb55f63-72f4-4470-b351-a2d0fc5df0ef None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:34.634960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238e0ad-e404-4bb6-8eba-4d5277914e67 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:34.635242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238e0ad-e404-4bb6-8eba-4d5277914e67 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:34.635519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238e0ad-e404-4bb6-8eba-4d5277914e67 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:34.636899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 133/260] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:34.657273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eae99-2d46-4621-bbb0-e16313fc5430 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:34.657273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eae99-2d46-4621-bbb0-e16313fc5430 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:34.657273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eae99-2d46-4621-bbb0-e16313fc5430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:34.683179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b110359-1fde-4683-b833-114c651f5cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:34.684470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b110359-1fde-4683-b833-114c651f5cce None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:34.684978 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b110359-1fde-4683-b833-114c651f5cce None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:34.685449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b110359-1fde-4683-b833-114c651f5cce None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:34.687435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b110359-1fde-4683-b833-114c651f5cce None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:34.688070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8f1eae99-2d46-4621-bbb0-e16313fc5430 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:34.691030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8f1eae99-2d46-4621-bbb0-e16313fc5430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:34.691260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b110359-1fde-4683-b833-114c651f5cce None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:34.695617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 128/261] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:34 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:34.705499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b110359-1fde-4683-b833-114c651f5cce None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:34.819568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 134/262] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:34.858811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1eae99-2d46-4621-bbb0-e16313fc5430 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:34.871810 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b110359-1fde-4683-b833-114c651f5cce None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:34.895676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:34.940771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffa6cb-1ffb-4736-b93c-ea19fb3a5f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:34.941729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffa6cb-1ffb-4736-b93c-ea19fb3a5f7c None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:34.942090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffa6cb-1ffb-4736-b93c-ea19fb3a5f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:34.942379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffa6cb-1ffb-4736-b93c-ea19fb3a5f7c None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7/users/1b64775d0c0f40869a3e56ebb647a5d3/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:34.945816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffa6cb-1ffb-4736-b93c-ea19fb3a5f7c None admin] RBAC: Authorizing `identity:create_grant(project_id=7e689a8addd941f8a2efd52508e178b7, user_id=1b64775d0c0f40869a3e56ebb647a5d3, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:34.947317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffa6cb-1ffb-4736-b93c-ea19fb3a5f7c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:34.968942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 129/263] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:01:34 2022] PUT /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7/users/1b64775d0c0f40869a3e56ebb647a5d3/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:34.974799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:35.021812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1eae99-2d46-4621-bbb0-e16313fc5430 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:35.076274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4dc9a-ba18-4cf0-9ec8-204fc224a71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:35.077030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4dc9a-ba18-4cf0-9ec8-204fc224a71b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:35.077318 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4dc9a-ba18-4cf0-9ec8-204fc224a71b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:35.077616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4dc9a-ba18-4cf0-9ec8-204fc224a71b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:35.092939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4dc9a-ba18-4cf0-9ec8-204fc224a71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-b5c4dc9a-ba18-4cf0-9ec8-204fc224a71b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:35.101796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:35.102788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:35.103577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 135/264] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:35.109711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffa6cb-1ffb-4736-b93c-ea19fb3a5f7c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:35.238789 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffa6cb-1ffb-4736-b93c-ea19fb3a5f7c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:35.274481 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d76cc-04fd-4013-9c7e-0faedb5b9f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:35.275165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d76cc-04fd-4013-9c7e-0faedb5b9f4d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:35.275455 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d76cc-04fd-4013-9c7e-0faedb5b9f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:35.275734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d76cc-04fd-4013-9c7e-0faedb5b9f4d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:35.276505 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d76cc-04fd-4013-9c7e-0faedb5b9f4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:35.276815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d76cc-04fd-4013-9c7e-0faedb5b9f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:35.278250 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d76cc-04fd-4013-9c7e-0faedb5b9f4d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:35.289570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 130/265] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:35.295275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4dc9a-ba18-4cf0-9ec8-204fc224a71b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:35.429010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f35cdb-387a-41e5-9b1a-bad9fe52e30a None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:35.429297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f35cdb-387a-41e5-9b1a-bad9fe52e30a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:35.429805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f35cdb-387a-41e5-9b1a-bad9fe52e30a None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:35.431608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 131/266] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:35.436781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6471bc-87d9-4c00-a7a9-0586ef27b7eb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:35.437309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6471bc-87d9-4c00-a7a9-0586ef27b7eb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:35.437782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6471bc-87d9-4c00-a7a9-0586ef27b7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:35.455258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-de6471bc-87d9-4c00-a7a9-0586ef27b7eb None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:35.456804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-de6471bc-87d9-4c00-a7a9-0586ef27b7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:35.465088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4dc9a-ba18-4cf0-9ec8-204fc224a71b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:35.489589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:35.524005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:35.533075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a6b2f-4c8a-43de-87f9-12e324e6bc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:35.533988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a6b2f-4c8a-43de-87f9-12e324e6bc22 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:35.534282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a6b2f-4c8a-43de-87f9-12e324e6bc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:35.534565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a6b2f-4c8a-43de-87f9-12e324e6bc22 None admin] PATH_INFO: `/v3/users/9e1438b660424e5db78809a8ea654dd0` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:35.536857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a6b2f-4c8a-43de-87f9-12e324e6bc22 None admin] RBAC: Authorizing `identity:get_user(user_id=9e1438b660424e5db78809a8ea654dd0)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:35.538367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a6b2f-4c8a-43de-87f9-12e324e6bc22 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:35.541304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 136/267] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:35 2022] GET /identity/v3/users/9e1438b660424e5db78809a8ea654dd0 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:35.547659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a6b2f-4c8a-43de-87f9-12e324e6bc22 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:35.570773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 132/268] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:35.596323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6471bc-87d9-4c00-a7a9-0586ef27b7eb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:35.685088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a6b2f-4c8a-43de-87f9-12e324e6bc22 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:35.720690 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a35255-36bd-4e12-9300-ba141b8e1464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:35.721584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35255-36bd-4e12-9300-ba141b8e1464 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:35.722048 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35255-36bd-4e12-9300-ba141b8e1464 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:35.722469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35255-36bd-4e12-9300-ba141b8e1464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:35.726055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6471bc-87d9-4c00-a7a9-0586ef27b7eb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:35.768921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2de1c3-214e-4f5c-b0cb-d33f8ab20172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:35.770031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2de1c3-214e-4f5c-b0cb-d33f8ab20172 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:35.770464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2de1c3-214e-4f5c-b0cb-d33f8ab20172 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:35.770880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2de1c3-214e-4f5c-b0cb-d33f8ab20172 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:35.772235 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2de1c3-214e-4f5c-b0cb-d33f8ab20172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:35.773960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2de1c3-214e-4f5c-b0cb-d33f8ab20172 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:35.775948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 133/269] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:35.777373 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d01081-20c9-4198-b833-f07bd94561c2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:35.777935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d01081-20c9-4198-b833-f07bd94561c2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:35.778409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d01081-20c9-4198-b833-f07bd94561c2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:35.780195 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 134/270] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:35.781898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d01081-20c9-4198-b833-f07bd94561c2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:35.796749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:35.800069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a35255-36bd-4e12-9300-ba141b8e1464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:35.801918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a35255-36bd-4e12-9300-ba141b8e1464 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:35.819046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:35.900103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d01081-20c9-4198-b833-f07bd94561c2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:35.906224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 137/271] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:35.907996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30202-b6b9-4b28-8a0a-c9c8ca6b61b0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:35.908468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30202-b6b9-4b28-8a0a-c9c8ca6b61b0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:35.908917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30202-b6b9-4b28-8a0a-c9c8ca6b61b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:35.925508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7ab30202-b6b9-4b28-8a0a-c9c8ca6b61b0 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:35.927070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7ab30202-b6b9-4b28-8a0a-c9c8ca6b61b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:35.937906 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dbba7-e514-4eb2-8057-aa7769630ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:35.938814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dbba7-e514-4eb2-8057-aa7769630ad4 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:35.939316 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dbba7-e514-4eb2-8057-aa7769630ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:35.939794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4dbba7-e514-4eb2-8057-aa7769630ad4 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:35.940716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4dbba7-e514-4eb2-8057-aa7769630ad4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:35.943057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4dbba7-e514-4eb2-8057-aa7769630ad4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:35.967370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 135/272] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:01:35 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:35.968535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dbba7-e514-4eb2-8057-aa7769630ad4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:36.013318 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 138/273] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:36.058758 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab30202-b6b9-4b28-8a0a-c9c8ca6b61b0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:36.091834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:36.123314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:36.164188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4dbba7-e514-4eb2-8057-aa7769630ad4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:36.185905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab30202-b6b9-4b28-8a0a-c9c8ca6b61b0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:36.224664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a4d04-b0bc-4aeb-99f8-95487d1a66ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:36.225447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a4d04-b0bc-4aeb-99f8-95487d1a66ba None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:36.225818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a4d04-b0bc-4aeb-99f8-95487d1a66ba None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:36.226621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc21e6-8648-4f88-8cf6-c60c65d70624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:36.227285 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a4d04-b0bc-4aeb-99f8-95487d1a66ba None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:36.227906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc21e6-8648-4f88-8cf6-c60c65d70624 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:36.228381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc21e6-8648-4f88-8cf6-c60c65d70624 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:36.228875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc21e6-8648-4f88-8cf6-c60c65d70624 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:36.229406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a4d04-b0bc-4aeb-99f8-95487d1a66ba None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:36.231181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a4d04-b0bc-4aeb-99f8-95487d1a66ba None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:36.233095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 136/274] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:35 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:36.239418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a4d04-b0bc-4aeb-99f8-95487d1a66ba None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:36.242640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc21e6-8648-4f88-8cf6-c60c65d70624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-e2fc21e6-8648-4f88-8cf6-c60c65d70624 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:36.251190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:36.252466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:36.253303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:36.256368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 139/275] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:36.263612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc21e6-8648-4f88-8cf6-c60c65d70624 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:36.356457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a4d04-b0bc-4aeb-99f8-95487d1a66ba None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:36.368193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:36.383666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc21e6-8648-4f88-8cf6-c60c65d70624 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:36.406462 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310ce43-a773-4e06-8090-f9491fde82db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:36.407368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310ce43-a773-4e06-8090-f9491fde82db None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:36.407795 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310ce43-a773-4e06-8090-f9491fde82db None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:36.408205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310ce43-a773-4e06-8090-f9491fde82db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:36.409182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310ce43-a773-4e06-8090-f9491fde82db None admin] RBAC: Adding query filter params (role.id=2eea147ab8b1426294a08106410f6d45, scope.project.id=7e689a8addd941f8a2efd52508e178b7, user.id=9e1438b660424e5db78809a8ea654dd0) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:36.409662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310ce43-a773-4e06-8090-f9491fde82db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:36.411469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310ce43-a773-4e06-8090-f9491fde82db None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:36.419258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 137/276] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:36 2022] GET /identity/v3/role_assignments?user.id=9e1438b660424e5db78809a8ea654dd0&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=2eea147ab8b1426294a08106410f6d45 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:36.420526 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a937-4705-4f60-b076-95f26a2d22b9 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:36.420820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a937-4705-4f60-b076-95f26a2d22b9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:36.421102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500a937-4705-4f60-b076-95f26a2d22b9 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:36.422361 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 138/277] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:36.428692 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56100b-5fc8-4978-b74e-3e30f295caff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:36.429018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720c6c8-a8be-4832-9765-145037683cf7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:36.429378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720c6c8-a8be-4832-9765-145037683cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:36.429727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720c6c8-a8be-4832-9765-145037683cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:36.430114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56100b-5fc8-4978-b74e-3e30f295caff None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:36.430530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56100b-5fc8-4978-b74e-3e30f295caff None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:36.430940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56100b-5fc8-4978-b74e-3e30f295caff None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:36.431857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56100b-5fc8-4978-b74e-3e30f295caff None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:36.432323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56100b-5fc8-4978-b74e-3e30f295caff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:36.433957 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56100b-5fc8-4978-b74e-3e30f295caff None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:36.443724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 140/278] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:36.450306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56100b-5fc8-4978-b74e-3e30f295caff None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:36.452548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7720c6c8-a8be-4832-9765-145037683cf7 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:36.453968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7720c6c8-a8be-4832-9765-145037683cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:36.478701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:36.553096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 139/279] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:36.586339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56100b-5fc8-4978-b74e-3e30f295caff None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:36.589032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720c6c8-a8be-4832-9765-145037683cf7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:36.631026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f20545-a096-4dd5-a83b-9a5ed7d805c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:36.632187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20545-a096-4dd5-a83b-9a5ed7d805c9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:36.632603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20545-a096-4dd5-a83b-9a5ed7d805c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:36.633022 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20545-a096-4dd5-a83b-9a5ed7d805c9 None admin] PATH_INFO: `/v3/users/5a2b33079a654d8e9ab8807afd23f335` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:36.649467 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f20545-a096-4dd5-a83b-9a5ed7d805c9 None admin] RBAC: Authorizing `identity:get_user(user_id=5a2b33079a654d8e9ab8807afd23f335)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:36.651226 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f20545-a096-4dd5-a83b-9a5ed7d805c9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:36.653155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 141/280] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:36 2022] GET /identity/v3/users/5a2b33079a654d8e9ab8807afd23f335 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:36.660585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f20545-a096-4dd5-a83b-9a5ed7d805c9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:36.674336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:36.712052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720c6c8-a8be-4832-9765-145037683cf7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:36.753128 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc95fed-0d5e-4956-9320-e43eff7e1818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:36.754508 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc95fed-0d5e-4956-9320-e43eff7e1818 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:36.754989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc95fed-0d5e-4956-9320-e43eff7e1818 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:36.755492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc95fed-0d5e-4956-9320-e43eff7e1818 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:36.767886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc95fed-0d5e-4956-9320-e43eff7e1818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-6dc95fed-0d5e-4956-9320-e43eff7e1818 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:36.778177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:36.779133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:36.779924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:36.781653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f20545-a096-4dd5-a83b-9a5ed7d805c9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:36.782628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 140/281] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:36.789244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc95fed-0d5e-4956-9320-e43eff7e1818 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:36.806290 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:36.851605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e81f41-80d3-428a-a779-2d6776b29120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:36.852906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81f41-80d3-428a-a779-2d6776b29120 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:36.853642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81f41-80d3-428a-a779-2d6776b29120 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:36.854574 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e81f41-80d3-428a-a779-2d6776b29120 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:36.855951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e81f41-80d3-428a-a779-2d6776b29120 None admin] RBAC: Adding query filter params (role.id=0b98291295724747acd01dd0d2306e62, user.id=5a2b33079a654d8e9ab8807afd23f335, scope.system=all) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:36.856729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e81f41-80d3-428a-a779-2d6776b29120 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:36.859112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e81f41-80d3-428a-a779-2d6776b29120 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:36.868803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 142/282] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 4 09:01:36 2022] GET /identity/v3/role_assignments?user.id=5a2b33079a654d8e9ab8807afd23f335&scope.system=all&role.id=0b98291295724747acd01dd0d2306e62 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:36.907059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc95fed-0d5e-4956-9320-e43eff7e1818 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:36.952574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1e271-6bbb-4d86-ba1b-ea7a09f0d624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:36.953654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1e271-6bbb-4d86-ba1b-ea7a09f0d624 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:36.954169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1e271-6bbb-4d86-ba1b-ea7a09f0d624 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:36.954658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1e271-6bbb-4d86-ba1b-ea7a09f0d624 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:36.955620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1e271-6bbb-4d86-ba1b-ea7a09f0d624 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:36.956125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1e271-6bbb-4d86-ba1b-ea7a09f0d624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:36.957918 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1e271-6bbb-4d86-ba1b-ea7a09f0d624 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:36.971894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 141/283] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:36.978170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e81f41-80d3-428a-a779-2d6776b29120 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:37.076207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:37.109912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e81f41-80d3-428a-a779-2d6776b29120 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:37.146783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e6deb-126d-49e1-b916-f513afde579d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:37.148064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e6deb-126d-49e1-b916-f513afde579d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:37.148549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e6deb-126d-49e1-b916-f513afde579d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:37.149026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e6deb-126d-49e1-b916-f513afde579d None admin] PATH_INFO: `/v3/users/3a51b27be6924ac58c85b3f110cfbf7e` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:37.151075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e6deb-126d-49e1-b916-f513afde579d None admin] RBAC: Authorizing `identity:get_user(user_id=3a51b27be6924ac58c85b3f110cfbf7e)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:37.152861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e6deb-126d-49e1-b916-f513afde579d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:37.154866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 143/284] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:36 2022] GET /identity/v3/users/3a51b27be6924ac58c85b3f110cfbf7e => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:37.161042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1e271-6bbb-4d86-ba1b-ea7a09f0d624 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:37.238791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:37.271193 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1e271-6bbb-4d86-ba1b-ea7a09f0d624 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:37.306112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38b1d0-9df8-4d5c-9cda-b99cd71c33e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:37.306991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38b1d0-9df8-4d5c-9cda-b99cd71c33e0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:37.307278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38b1d0-9df8-4d5c-9cda-b99cd71c33e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:37.307555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38b1d0-9df8-4d5c-9cda-b99cd71c33e0 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:37.308762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38b1d0-9df8-4d5c-9cda-b99cd71c33e0 None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:37.310257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38b1d0-9df8-4d5c-9cda-b99cd71c33e0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:37.311987 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 142/285] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:37 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:37.318096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e6deb-126d-49e1-b916-f513afde579d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:37.421898 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e6deb-126d-49e1-b916-f513afde579d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:37.457118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee437812-f6dd-4443-bb17-a517b57ad5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:37.458340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee437812-f6dd-4443-bb17-a517b57ad5a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:37.458626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee437812-f6dd-4443-bb17-a517b57ad5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:37.458904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee437812-f6dd-4443-bb17-a517b57ad5a6 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7/users/3a51b27be6924ac58c85b3f110cfbf7e/roles/67473bdffb674c6b8d0afd413642eff6` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:37.460889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee437812-f6dd-4443-bb17-a517b57ad5a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e689a8addd941f8a2efd52508e178b7, user_id=3a51b27be6924ac58c85b3f110cfbf7e, role_id=67473bdffb674c6b8d0afd413642eff6)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:37.462429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee437812-f6dd-4443-bb17-a517b57ad5a6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:37.472399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0eb143-6f6a-4db8-9f19-9933913a3e33 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:37.472772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0eb143-6f6a-4db8-9f19-9933913a3e33 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:37.473029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0eb143-6f6a-4db8-9f19-9933913a3e33 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:37.474623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 143/286] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:37.477588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:37.481543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 144/287] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:01:37 2022] PUT /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7/users/3a51b27be6924ac58c85b3f110cfbf7e/roles/67473bdffb674c6b8d0afd413642eff6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:37.487495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bda38-65a4-4461-b988-916bba41368b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:37.487765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bda38-65a4-4461-b988-916bba41368b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:37.488106 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bda38-65a4-4461-b988-916bba41368b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:37.510996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-601bda38-65a4-4461-b988-916bba41368b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:37.512223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-601bda38-65a4-4461-b988-916bba41368b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:37.585574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 144/288] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:37.621165 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee437812-f6dd-4443-bb17-a517b57ad5a6 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:37.759959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee437812-f6dd-4443-bb17-a517b57ad5a6 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:37.800296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb0eec-9413-4d16-8f17-367f8d01e964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:37.801278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb0eec-9413-4d16-8f17-367f8d01e964 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:37.801570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb0eec-9413-4d16-8f17-367f8d01e964 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:37.801844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb0eec-9413-4d16-8f17-367f8d01e964 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:37.813188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb0eec-9413-4d16-8f17-367f8d01e964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5ccb0eec-9413-4d16-8f17-367f8d01e964 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:37.821333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:37.822345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:37.823161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 145/289] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:37.830145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bda38-65a4-4461-b988-916bba41368b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:37.906237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:37.936480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bda38-65a4-4461-b988-916bba41368b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:37.973573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abd1f2-e825-49e6-af91-affdf0efba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:37.974290 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd1f2-e825-49e6-af91-affdf0efba8d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:37.974576 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd1f2-e825-49e6-af91-affdf0efba8d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:37.974855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd1f2-e825-49e6-af91-affdf0efba8d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:37.975619 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abd1f2-e825-49e6-af91-affdf0efba8d None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:37.975982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abd1f2-e825-49e6-af91-affdf0efba8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:37.977393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abd1f2-e825-49e6-af91-affdf0efba8d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:37.989386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 145/290] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:37.992215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb0eec-9413-4d16-8f17-367f8d01e964 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:38.055154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:38.109523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb0eec-9413-4d16-8f17-367f8d01e964 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:38.167584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b1423-9c5e-400e-8b98-bef0b25dc914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:38.168476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b1423-9c5e-400e-8b98-bef0b25dc914 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:38.168765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b1423-9c5e-400e-8b98-bef0b25dc914 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:38.169050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b1423-9c5e-400e-8b98-bef0b25dc914 None admin] PATH_INFO: `/v3/users/1b64775d0c0f40869a3e56ebb647a5d3` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:38.170298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b1423-9c5e-400e-8b98-bef0b25dc914 None admin] RBAC: Authorizing `identity:get_user(user_id=1b64775d0c0f40869a3e56ebb647a5d3)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:38.171817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b1423-9c5e-400e-8b98-bef0b25dc914 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:38.173638 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 146/291] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:37 2022] GET /identity/v3/users/1b64775d0c0f40869a3e56ebb647a5d3 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:38.179272 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abd1f2-e825-49e6-af91-affdf0efba8d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:38.294417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abd1f2-e825-49e6-af91-affdf0efba8d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:38.325209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:38.334330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e304e5a-c420-479a-a11d-6f721fb6c429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:38.335025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e304e5a-c420-479a-a11d-6f721fb6c429 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:38.335319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e304e5a-c420-479a-a11d-6f721fb6c429 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:38.335604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e304e5a-c420-479a-a11d-6f721fb6c429 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:38.411704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e304e5a-c420-479a-a11d-6f721fb6c429 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:38.413187 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e304e5a-c420-479a-a11d-6f721fb6c429 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:38.519604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 146/292] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:38.521090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b1423-9c5e-400e-8b98-bef0b25dc914 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:38.648542 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b1423-9c5e-400e-8b98-bef0b25dc914 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:38.685712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:38.693841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b0273-7561-4e95-8ad5-22e5c6e20a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:38.694742 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b0273-7561-4e95-8ad5-22e5c6e20a3b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:38.695041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b0273-7561-4e95-8ad5-22e5c6e20a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:38.698240 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b0273-7561-4e95-8ad5-22e5c6e20a3b None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:38.699476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b0273-7561-4e95-8ad5-22e5c6e20a3b None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:38.700953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b0273-7561-4e95-8ad5-22e5c6e20a3b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:38.702794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 147/293] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:38 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:38.708948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e304e5a-c420-479a-a11d-6f721fb6c429 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:38.749165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:38.837916 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c331c-9703-4bbe-9524-e695b1319e02 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:38.838260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c331c-9703-4bbe-9524-e695b1319e02 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:38.838478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c331c-9703-4bbe-9524-e695b1319e02 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:38.839868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 148/294] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:38.844862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da508b-5a2f-4412-b153-eaa6e89f8c44 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:38.845153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da508b-5a2f-4412-b153-eaa6e89f8c44 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:38.845447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da508b-5a2f-4412-b153-eaa6e89f8c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:38.846874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e304e5a-c420-479a-a11d-6f721fb6c429 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:38.872765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-52da508b-5a2f-4412-b153-eaa6e89f8c44 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:38.873983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-52da508b-5a2f-4412-b153-eaa6e89f8c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:38.909728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48f647-800a-4830-99ad-479b5724935f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:38.910623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f647-800a-4830-99ad-479b5724935f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:38.910997 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f647-800a-4830-99ad-479b5724935f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:38.911200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f647-800a-4830-99ad-479b5724935f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:38.912049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48f647-800a-4830-99ad-479b5724935f None admin] RBAC: Adding query filter params (role.id=0b98291295724747acd01dd0d2306e62, scope.project.id=7e689a8addd941f8a2efd52508e178b7, user.id=1b64775d0c0f40869a3e56ebb647a5d3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:38.912366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48f647-800a-4830-99ad-479b5724935f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:38.913833 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48f647-800a-4830-99ad-479b5724935f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:38.926633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 147/295] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:38 2022] GET /identity/v3/role_assignments?user.id=1b64775d0c0f40869a3e56ebb647a5d3&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=0b98291295724747acd01dd0d2306e62 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:39.011734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 149/296] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 04 09:01:39.035232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48f647-800a-4830-99ad-479b5724935f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:39.138024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:39.147933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48f647-800a-4830-99ad-479b5724935f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:39.183373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe57a8-0a0a-48dd-ae26-38929392c485 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:39.183980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe57a8-0a0a-48dd-ae26-38929392c485 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:39.184451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe57a8-0a0a-48dd-ae26-38929392c485 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:39.184717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaba250-d4c8-4280-94be-37e16e235248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:39.185560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaba250-d4c8-4280-94be-37e16e235248 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:39.185842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaba250-d4c8-4280-94be-37e16e235248 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:39.186169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaba250-d4c8-4280-94be-37e16e235248 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:39.187407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaba250-d4c8-4280-94be-37e16e235248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:39.187608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 150/297] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:39.189915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaba250-d4c8-4280-94be-37e16e235248 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:39.192305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 148/298] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:39.193864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce0b89-1e63-466f-9d6e-9948defcfea5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:39.194278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce0b89-1e63-466f-9d6e-9948defcfea5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:39.194621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce0b89-1e63-466f-9d6e-9948defcfea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:39.197805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaba250-d4c8-4280-94be-37e16e235248 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:39.213621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-71ce0b89-1e63-466f-9d6e-9948defcfea5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:39.215138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-71ce0b89-1e63-466f-9d6e-9948defcfea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:39.294103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 151/299] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:39.328790 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce0b89-1e63-466f-9d6e-9948defcfea5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:39.338218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaba250-d4c8-4280-94be-37e16e235248 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:39.377949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2860a4-7d55-405f-9279-a9c3f70d9f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:39.378930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2860a4-7d55-405f-9279-a9c3f70d9f6f None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:39.379421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2860a4-7d55-405f-9279-a9c3f70d9f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:39.379867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2860a4-7d55-405f-9279-a9c3f70d9f6f None admin] PATH_INFO: `/v3/groups` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:39.380764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2860a4-7d55-405f-9279-a9c3f70d9f6f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:39.382483 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2860a4-7d55-405f-9279-a9c3f70d9f6f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:39.399138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:39.404755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 149/300] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Jul 4 09:01:39 2022] POST /identity/v3/groups => generated 229 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:01:39.459932 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce0b89-1e63-466f-9d6e-9948defcfea5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:39.500615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373e8bb-3315-40b0-b794-43685c96e78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:39.502158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373e8bb-3315-40b0-b794-43685c96e78a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:39.502635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373e8bb-3315-40b0-b794-43685c96e78a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:39.503043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373e8bb-3315-40b0-b794-43685c96e78a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:39.515945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373e8bb-3315-40b0-b794-43685c96e78a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-9373e8bb-3315-40b0-b794-43685c96e78a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:39.524455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:39.526086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:39.527083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:39.529198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 152/301] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:39.535281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2860a4-7d55-405f-9279-a9c3f70d9f6f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:39.569012 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:39.592926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c343e-38fe-498a-a8ce-5620cd535ea9 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:39.593569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c343e-38fe-498a-a8ce-5620cd535ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:39.594013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c343e-38fe-498a-a8ce-5620cd535ea9 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:39.595670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 153/302] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:39.601407 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e329fd4-9e2c-4270-9b9e-ab22dd875967 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:39.601685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e329fd4-9e2c-4270-9b9e-ab22dd875967 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:39.602000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e329fd4-9e2c-4270-9b9e-ab22dd875967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:39.623524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5e329fd4-9e2c-4270-9b9e-ab22dd875967 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:39.624740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5e329fd4-9e2c-4270-9b9e-ab22dd875967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:39.666959 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2860a4-7d55-405f-9279-a9c3f70d9f6f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:39.706488 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 154/303] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:39.715634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbfd6e-cafb-4c0b-b6a7-a3efb3a68b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:39.716252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbfd6e-cafb-4c0b-b6a7-a3efb3a68b50 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:39.716537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbfd6e-cafb-4c0b-b6a7-a3efb3a68b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:39.716814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbfd6e-cafb-4c0b-b6a7-a3efb3a68b50 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:39.717568 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbfd6e-cafb-4c0b-b6a7-a3efb3a68b50 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:39.717873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbfd6e-cafb-4c0b-b6a7-a3efb3a68b50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:39.719287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbfd6e-cafb-4c0b-b6a7-a3efb3a68b50 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:39.728236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 150/304] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:39.733781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e329fd4-9e2c-4270-9b9e-ab22dd875967 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:39.736791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbfd6e-cafb-4c0b-b6a7-a3efb3a68b50 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:39.784099 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:39.847703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e329fd4-9e2c-4270-9b9e-ab22dd875967 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:39.855227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:39.875108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbfd6e-cafb-4c0b-b6a7-a3efb3a68b50 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:39.893003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5c173-0d53-4d47-b107-0f1b5f807755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:39.893962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5c173-0d53-4d47-b107-0f1b5f807755 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:39.894246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5c173-0d53-4d47-b107-0f1b5f807755 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:39.894571 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5c173-0d53-4d47-b107-0f1b5f807755 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:39.906232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5c173-0d53-4d47-b107-0f1b5f807755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:39.912527 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d40ee-45d2-4c08-a3a7-ca7e94d2068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:39.913653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d40ee-45d2-4c08-a3a7-ca7e94d2068e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:39.914187 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d40ee-45d2-4c08-a3a7-ca7e94d2068e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-67f5c173-0d53-4d47-b107-0f1b5f807755 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:39.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:39.915486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:39.916370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 155/305] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:39.922360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d40ee-45d2-4c08-a3a7-ca7e94d2068e None admin] PATH_INFO: `/v3/users/866b9a3a2e2b4575adb018eeedf06285` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:39.923697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5c173-0d53-4d47-b107-0f1b5f807755 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:39.936383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d40ee-45d2-4c08-a3a7-ca7e94d2068e None admin] RBAC: Authorizing `identity:get_user(user_id=866b9a3a2e2b4575adb018eeedf06285)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:39.938409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d40ee-45d2-4c08-a3a7-ca7e94d2068e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:39.940323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 151/306] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:39 2022] GET /identity/v3/users/866b9a3a2e2b4575adb018eeedf06285 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:39.945869 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d40ee-45d2-4c08-a3a7-ca7e94d2068e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:40.052529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5c173-0d53-4d47-b107-0f1b5f807755 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:40.063064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:40.063413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d40ee-45d2-4c08-a3a7-ca7e94d2068e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:40.099469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a51e0-ee7e-47c1-89cb-81ea6c2b9aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:40.100126 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a51e0-ee7e-47c1-89cb-81ea6c2b9aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:40.100405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a51e0-ee7e-47c1-89cb-81ea6c2b9aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:40.100676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a51e0-ee7e-47c1-89cb-81ea6c2b9aa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:40.101515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a51e0-ee7e-47c1-89cb-81ea6c2b9aa3 None admin] RBAC: Adding query filter params (role.id=67473bdffb674c6b8d0afd413642eff6, user.id=866b9a3a2e2b4575adb018eeedf06285, scope.system=all) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:40.101826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a51e0-ee7e-47c1-89cb-81ea6c2b9aa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:40.102179 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3910d-8fb2-469d-888b-211989635166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:40.103066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3910d-8fb2-469d-888b-211989635166 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:40.103312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a51e0-ee7e-47c1-89cb-81ea6c2b9aa3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:40.103731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3910d-8fb2-469d-888b-211989635166 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:40.104193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3910d-8fb2-469d-888b-211989635166 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:40.105396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3910d-8fb2-469d-888b-211989635166 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:40.106290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3910d-8fb2-469d-888b-211989635166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:40.107663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3910d-8fb2-469d-888b-211989635166 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:40.111166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 152/307] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 4 09:01:39 2022] GET /identity/v3/role_assignments?user.id=866b9a3a2e2b4575adb018eeedf06285&scope.system=all&role.id=67473bdffb674c6b8d0afd413642eff6 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:40.122079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 156/308] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:40.128380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a51e0-ee7e-47c1-89cb-81ea6c2b9aa3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:40.141909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:40.243087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a51e0-ee7e-47c1-89cb-81ea6c2b9aa3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:40.280263 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447113a1-75cb-446b-aee2-eae0be36a63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:40.281220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447113a1-75cb-446b-aee2-eae0be36a63b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:40.281506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447113a1-75cb-446b-aee2-eae0be36a63b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:40.281781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447113a1-75cb-446b-aee2-eae0be36a63b None admin] PATH_INFO: `/v3/users/5a2b33079a654d8e9ab8807afd23f335` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:40.283049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447113a1-75cb-446b-aee2-eae0be36a63b None admin] RBAC: Authorizing `identity:get_user(user_id=5a2b33079a654d8e9ab8807afd23f335)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:40.284510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447113a1-75cb-446b-aee2-eae0be36a63b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:40.286728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 153/309] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:40 2022] GET /identity/v3/users/5a2b33079a654d8e9ab8807afd23f335 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:40.292378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3910d-8fb2-469d-888b-211989635166 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:40.303500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ca8fb-ac29-4c3e-97f4-c595b9de4f76 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:40.304069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ca8fb-ac29-4c3e-97f4-c595b9de4f76 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:40.304530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ca8fb-ac29-4c3e-97f4-c595b9de4f76 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:40.306099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 154/310] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:40.311129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317a372-d724-44fd-a862-6212e3d6b262 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:40.311419 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317a372-d724-44fd-a862-6212e3d6b262 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:40.311706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317a372-d724-44fd-a862-6212e3d6b262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:40.329130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d317a372-d724-44fd-a862-6212e3d6b262 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:40.330363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d317a372-d724-44fd-a862-6212e3d6b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:40.406240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 155/311] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:40.412595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3910d-8fb2-469d-888b-211989635166 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:40.426181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317a372-d724-44fd-a862-6212e3d6b262 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:40.435056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:40.448084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842377d9-3143-493f-bfd8-7d38ba98f812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:40.449000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842377d9-3143-493f-bfd8-7d38ba98f812 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:40.449333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842377d9-3143-493f-bfd8-7d38ba98f812 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:40.449614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842377d9-3143-493f-bfd8-7d38ba98f812 None admin] PATH_INFO: `/v3/system/users/5a2b33079a654d8e9ab8807afd23f335/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:40.451162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842377d9-3143-493f-bfd8-7d38ba98f812 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5a2b33079a654d8e9ab8807afd23f335, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:40.452696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842377d9-3143-493f-bfd8-7d38ba98f812 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:40.465791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 157/312] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Jul 4 09:01:40 2022] PUT /identity/v3/system/users/5a2b33079a654d8e9ab8807afd23f335/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:40.539046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317a372-d724-44fd-a862-6212e3d6b262 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:40.577855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772b8ec-4581-445c-b636-462410c7855c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:40.579051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772b8ec-4581-445c-b636-462410c7855c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:40.579523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772b8ec-4581-445c-b636-462410c7855c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:40.579984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772b8ec-4581-445c-b636-462410c7855c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:40.592255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772b8ec-4581-445c-b636-462410c7855c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d772b8ec-4581-445c-b636-462410c7855c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:40.600504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:40.601460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:40.602284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:40.605092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 156/313] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:40.611293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842377d9-3143-493f-bfd8-7d38ba98f812 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:40.649804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:40.721448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842377d9-3143-493f-bfd8-7d38ba98f812 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:40.755460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e7b96-8ff9-4858-93a7-8889dd192005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:40.756348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e7b96-8ff9-4858-93a7-8889dd192005 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:40.756768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e7b96-8ff9-4858-93a7-8889dd192005 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:40.757212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e7b96-8ff9-4858-93a7-8889dd192005 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:40.758135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e7b96-8ff9-4858-93a7-8889dd192005 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:40.758577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e7b96-8ff9-4858-93a7-8889dd192005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:40.760080 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e7b96-8ff9-4858-93a7-8889dd192005 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:40.769127 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 158/314] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:40.774930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772b8ec-4581-445c-b636-462410c7855c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:40.841432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:40.876648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772b8ec-4581-445c-b636-462410c7855c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:40.910796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1afe27-3162-4728-9ff9-8baeb4a40bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:40.911716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1afe27-3162-4728-9ff9-8baeb4a40bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:40.912032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1afe27-3162-4728-9ff9-8baeb4a40bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:40.912315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1afe27-3162-4728-9ff9-8baeb4a40bb6 None admin] PATH_INFO: `/v3/users/3a51b27be6924ac58c85b3f110cfbf7e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:40.913500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1afe27-3162-4728-9ff9-8baeb4a40bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=3a51b27be6924ac58c85b3f110cfbf7e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:40.915056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1afe27-3162-4728-9ff9-8baeb4a40bb6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:40.916743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 157/315] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:40 2022] GET /identity/v3/users/3a51b27be6924ac58c85b3f110cfbf7e => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:40.922313 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e7b96-8ff9-4858-93a7-8889dd192005 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:41.020861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:41.027152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e7b96-8ff9-4858-93a7-8889dd192005 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:41.061894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd3297-b4d2-4bc1-997f-a6fdbde5c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:41.062557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd3297-b4d2-4bc1-997f-a6fdbde5c780 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:41.062837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd3297-b4d2-4bc1-997f-a6fdbde5c780 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:41.063155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd3297-b4d2-4bc1-997f-a6fdbde5c780 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:41.136818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd3297-b4d2-4bc1-997f-a6fdbde5c780 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:41.138239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd3297-b4d2-4bc1-997f-a6fdbde5c780 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:41.243148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 159/316] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:41.246268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:41.248779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1afe27-3162-4728-9ff9-8baeb4a40bb6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:41.373363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1afe27-3162-4728-9ff9-8baeb4a40bb6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:41.407008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246d8d7-d448-46ec-a98b-83a9a2165a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:41.407799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246d8d7-d448-46ec-a98b-83a9a2165a00 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:41.408080 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246d8d7-d448-46ec-a98b-83a9a2165a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:41.408351 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246d8d7-d448-46ec-a98b-83a9a2165a00 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:41.409526 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246d8d7-d448-46ec-a98b-83a9a2165a00 None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:41.411037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246d8d7-d448-46ec-a98b-83a9a2165a00 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:41.413045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 158/317] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:41 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:41.418825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd3297-b4d2-4bc1-997f-a6fdbde5c780 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:41.517912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd3297-b4d2-4bc1-997f-a6fdbde5c780 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:41.547783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9e4ff-a491-47ea-a1ba-1063541fe472 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:41.548067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9e4ff-a491-47ea-a1ba-1063541fe472 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:41.548343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9e4ff-a491-47ea-a1ba-1063541fe472 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:41.549716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 159/318] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:41.554869 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100e1c8-b2fb-4607-a0a0-b7c766c07ed8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:41.555153 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100e1c8-b2fb-4607-a0a0-b7c766c07ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:41.555426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100e1c8-b2fb-4607-a0a0-b7c766c07ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:41.560538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa3bb3-b9c7-4a8a-aab9-950a294f6645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:41.561215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa3bb3-b9c7-4a8a-aab9-950a294f6645 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:41.561501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa3bb3-b9c7-4a8a-aab9-950a294f6645 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:41.561779 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa3bb3-b9c7-4a8a-aab9-950a294f6645 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:41.562058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:41.562687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa3bb3-b9c7-4a8a-aab9-950a294f6645 None admin] RBAC: Adding query filter params (role.id=67473bdffb674c6b8d0afd413642eff6, scope.project.id=7e689a8addd941f8a2efd52508e178b7, user.id=3a51b27be6924ac58c85b3f110cfbf7e) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:41.563005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa3bb3-b9c7-4a8a-aab9-950a294f6645 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:41.564462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa3bb3-b9c7-4a8a-aab9-950a294f6645 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:41.571040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 160/319] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:01:41 2022] GET /identity/v3/role_assignments?user.id=3a51b27be6924ac58c85b3f110cfbf7e&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=67473bdffb674c6b8d0afd413642eff6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:41.583364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8100e1c8-b2fb-4607-a0a0-b7c766c07ed8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:41.584683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8100e1c8-b2fb-4607-a0a0-b7c766c07ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:41.672257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 160/320] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:41.695275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa3bb3-b9c7-4a8a-aab9-950a294f6645 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:41.757796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:41.801990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa3bb3-b9c7-4a8a-aab9-950a294f6645 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:41.836982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621742d-ff6c-4a7d-a670-0deb5c0e0193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:41.837789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621742d-ff6c-4a7d-a670-0deb5c0e0193 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:41.838120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621742d-ff6c-4a7d-a670-0deb5c0e0193 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:41.838378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621742d-ff6c-4a7d-a670-0deb5c0e0193 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:41.839576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621742d-ff6c-4a7d-a670-0deb5c0e0193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:41.841035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621742d-ff6c-4a7d-a670-0deb5c0e0193 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:41.843047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 161/321] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:01:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:41.848808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100e1c8-b2fb-4607-a0a0-b7c766c07ed8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:41.948296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100e1c8-b2fb-4607-a0a0-b7c766c07ed8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:41.979177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:41.985525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03825c84-38e2-4eaa-90ea-92aacd008f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:41.986262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03825c84-38e2-4eaa-90ea-92aacd008f89 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:41.986540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03825c84-38e2-4eaa-90ea-92aacd008f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:41.986826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03825c84-38e2-4eaa-90ea-92aacd008f89 None admin] PATH_INFO: `/v3/groups` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:41.987520 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03825c84-38e2-4eaa-90ea-92aacd008f89 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:41.988996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03825c84-38e2-4eaa-90ea-92aacd008f89 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:42.009917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 161/322] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Jul 4 09:01:41 2022] POST /identity/v3/groups => generated 226 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:01:42.235683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0301f-de3b-43df-bcc7-519f5371aa95 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:42.235898 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0301f-de3b-43df-bcc7-519f5371aa95 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:42.236148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0301f-de3b-43df-bcc7-519f5371aa95 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:42.237717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 162/323] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:42.243104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f399ac-0526-4cf0-a405-842318564966 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:42.243392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f399ac-0526-4cf0-a405-842318564966 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:42.243675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f399ac-0526-4cf0-a405-842318564966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:42.262561 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-75f399ac-0526-4cf0-a405-842318564966 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:42.264502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-75f399ac-0526-4cf0-a405-842318564966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:42.342824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 162/324] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:42.369944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0301f-de3b-43df-bcc7-519f5371aa95 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:42.479671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0301f-de3b-43df-bcc7-519f5371aa95 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:42.506982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:42.516690 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d68dcf-ba21-45d5-81a7-a2b8bb655ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:42.517528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d68dcf-ba21-45d5-81a7-a2b8bb655ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:42.517809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d68dcf-ba21-45d5-81a7-a2b8bb655ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:42.518112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d68dcf-ba21-45d5-81a7-a2b8bb655ca2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:42.530116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d68dcf-ba21-45d5-81a7-a2b8bb655ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d4d68dcf-ba21-45d5-81a7-a2b8bb655ca2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:42.538138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:42.539041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:42.540279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 163/325] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:42.545593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f399ac-0526-4cf0-a405-842318564966 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:42.622978 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70051f7d-b5aa-4485-8c32-bc1716fcbf69 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:42.623321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70051f7d-b5aa-4485-8c32-bc1716fcbf69 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:42.623536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70051f7d-b5aa-4485-8c32-bc1716fcbf69 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:42.625053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 164/326] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:42.631180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d8fe1-da25-47b8-9d51-f6936061dfe3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:42.631469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d8fe1-da25-47b8-9d51-f6936061dfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:42.631784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d8fe1-da25-47b8-9d51-f6936061dfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:42.656076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5d6d8fe1-da25-47b8-9d51-f6936061dfe3 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:42.657391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5d6d8fe1-da25-47b8-9d51-f6936061dfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:42.661870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f399ac-0526-4cf0-a405-842318564966 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:42.688365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:42.702146 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829c7eb-7d82-4893-ac81-29f36d32f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:42.703044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829c7eb-7d82-4893-ac81-29f36d32f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:42.703460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829c7eb-7d82-4893-ac81-29f36d32f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:42.703954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829c7eb-7d82-4893-ac81-29f36d32f6c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:42.705086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829c7eb-7d82-4893-ac81-29f36d32f6c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:42.705528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829c7eb-7d82-4893-ac81-29f36d32f6c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:42.707282 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829c7eb-7d82-4893-ac81-29f36d32f6c6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:42.716413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 163/327] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:42.724640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829c7eb-7d82-4893-ac81-29f36d32f6c6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:42.756931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 165/328] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:42.781832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d8fe1-da25-47b8-9d51-f6936061dfe3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:42.856555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829c7eb-7d82-4893-ac81-29f36d32f6c6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:42.906313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aeaaf4-5abc-4576-b796-533d09632eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:42.907572 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeaaf4-5abc-4576-b796-533d09632eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:42.908078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeaaf4-5abc-4576-b796-533d09632eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:42.908546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeaaf4-5abc-4576-b796-533d09632eb3 None admin] PATH_INFO: `/v3/users/866b9a3a2e2b4575adb018eeedf06285` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:42.910094 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aeaaf4-5abc-4576-b796-533d09632eb3 None admin] RBAC: Authorizing `identity:get_user(user_id=866b9a3a2e2b4575adb018eeedf06285)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:42.912210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aeaaf4-5abc-4576-b796-533d09632eb3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:42.914439 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 164/329] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:42 2022] GET /identity/v3/users/866b9a3a2e2b4575adb018eeedf06285 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:42.916095 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d8fe1-da25-47b8-9d51-f6936061dfe3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:42.929427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aeaaf4-5abc-4576-b796-533d09632eb3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:42.939122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:42.962142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265539f-99b2-4d2f-b33f-e416c14f60da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:42.963419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265539f-99b2-4d2f-b33f-e416c14f60da None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:42.963718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265539f-99b2-4d2f-b33f-e416c14f60da None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:42.963992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265539f-99b2-4d2f-b33f-e416c14f60da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:42.977773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265539f-99b2-4d2f-b33f-e416c14f60da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-2265539f-99b2-4d2f-b33f-e416c14f60da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:42.988504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:42.989585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:42.990445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 166/330] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:42.994511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:43.002568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265539f-99b2-4d2f-b33f-e416c14f60da None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:43.059630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aeaaf4-5abc-4576-b796-533d09632eb3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:43.134436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265539f-99b2-4d2f-b33f-e416c14f60da None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:43.143176 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c99ed-f086-43e3-8b06-2860a4abfe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:43.144087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c99ed-f086-43e3-8b06-2860a4abfe3a None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:43.144358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c99ed-f086-43e3-8b06-2860a4abfe3a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:43.144718 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c99ed-f086-43e3-8b06-2860a4abfe3a None admin] PATH_INFO: `/v3/system/users/866b9a3a2e2b4575adb018eeedf06285/roles/67473bdffb674c6b8d0afd413642eff6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:43.146380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c99ed-f086-43e3-8b06-2860a4abfe3a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=866b9a3a2e2b4575adb018eeedf06285, role_id=67473bdffb674c6b8d0afd413642eff6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:43.147778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c99ed-f086-43e3-8b06-2860a4abfe3a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:43.169255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 165/331] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Jul 4 09:01:42 2022] PUT /identity/v3/system/users/866b9a3a2e2b4575adb018eeedf06285/roles/67473bdffb674c6b8d0afd413642eff6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:01:43.191417 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52909a52-6f10-4141-844d-8acdb2a9b38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:43.192661 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52909a52-6f10-4141-844d-8acdb2a9b38a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:43.193158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52909a52-6f10-4141-844d-8acdb2a9b38a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:43.193631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52909a52-6f10-4141-844d-8acdb2a9b38a None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:43.194671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52909a52-6f10-4141-844d-8acdb2a9b38a None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:43.195164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52909a52-6f10-4141-844d-8acdb2a9b38a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:43.196805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52909a52-6f10-4141-844d-8acdb2a9b38a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:43.212675 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 167/332] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:43.218346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c99ed-f086-43e3-8b06-2860a4abfe3a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:43.352885 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c99ed-f086-43e3-8b06-2860a4abfe3a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:43.374386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:43.402703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d36a0-7fa0-474d-9e0f-27d75c060150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:43.403622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d36a0-7fa0-474d-9e0f-27d75c060150 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:43.403905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d36a0-7fa0-474d-9e0f-27d75c060150 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:43.404181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d36a0-7fa0-474d-9e0f-27d75c060150 None admin] PATH_INFO: `/v3/users/5a2b33079a654d8e9ab8807afd23f335` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:43.405566 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d36a0-7fa0-474d-9e0f-27d75c060150 None admin] RBAC: Authorizing `identity:get_user(user_id=5a2b33079a654d8e9ab8807afd23f335)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:43.407109 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d36a0-7fa0-474d-9e0f-27d75c060150 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:43.409160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 166/333] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:43 2022] GET /identity/v3/users/5a2b33079a654d8e9ab8807afd23f335 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:43.415464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52909a52-6f10-4141-844d-8acdb2a9b38a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:43.466009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:43.547982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52909a52-6f10-4141-844d-8acdb2a9b38a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:43.590853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8495a6-f83b-42da-8c62-a8b70936c4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:43.591885 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8495a6-f83b-42da-8c62-a8b70936c4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:43.592380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8495a6-f83b-42da-8c62-a8b70936c4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:43.592840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8495a6-f83b-42da-8c62-a8b70936c4a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:43.593861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8495a6-f83b-42da-8c62-a8b70936c4a7 None admin] RBAC: Adding query filter params (role.id=0b98291295724747acd01dd0d2306e62, user.id=5a2b33079a654d8e9ab8807afd23f335, scope.system=all) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:43.594384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8495a6-f83b-42da-8c62-a8b70936c4a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:43.595999 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8495a6-f83b-42da-8c62-a8b70936c4a7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:43.610658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 168/334] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 4 09:01:43 2022] GET /identity/v3/role_assignments?user.id=5a2b33079a654d8e9ab8807afd23f335&scope.system=all&role.id=0b98291295724747acd01dd0d2306e62 => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:45.071032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb492cd7-d134-41a4-96fb-5437bb46f65c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:45.072068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb492cd7-d134-41a4-96fb-5437bb46f65c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:45.072546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb492cd7-d134-41a4-96fb-5437bb46f65c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:45.074512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 167/335] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:45.079605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d58ff-6496-4206-a749-adb701fed4ee None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:45.080430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d58ff-6496-4206-a749-adb701fed4ee None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:45.080892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d58ff-6496-4206-a749-adb701fed4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:45.102136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f94d58ff-6496-4206-a749-adb701fed4ee None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:45.103479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29160fdc-153e-4461-bb87-128f40fefeb5 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:45.103778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29160fdc-153e-4461-bb87-128f40fefeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:45.104068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29160fdc-153e-4461-bb87-128f40fefeb5 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:45.104317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f94d58ff-6496-4206-a749-adb701fed4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:45.105803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 168/336] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:45.110423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638186cd-3041-4c1f-b1ad-7d532318bc9d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:45.110717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638186cd-3041-4c1f-b1ad-7d532318bc9d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:45.110998 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638186cd-3041-4c1f-b1ad-7d532318bc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:45.141733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-638186cd-3041-4c1f-b1ad-7d532318bc9d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:45.143004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-638186cd-3041-4c1f-b1ad-7d532318bc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:45.266723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 169/337] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:45.267980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 169/338] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:45.302607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638186cd-3041-4c1f-b1ad-7d532318bc9d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:45.315967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d58ff-6496-4206-a749-adb701fed4ee None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:45.362938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:45.432453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638186cd-3041-4c1f-b1ad-7d532318bc9d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:45.450170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:45.499772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d58ff-6496-4206-a749-adb701fed4ee None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:45.499997 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac65e03-6c08-4a18-8e68-bdb79a3f20f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:45.500721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac65e03-6c08-4a18-8e68-bdb79a3f20f5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:45.501009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac65e03-6c08-4a18-8e68-bdb79a3f20f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:45.501290 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac65e03-6c08-4a18-8e68-bdb79a3f20f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:45.531806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac65e03-6c08-4a18-8e68-bdb79a3f20f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-eac65e03-6c08-4a18-8e68-bdb79a3f20f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:45.539731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:45.540641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:45.541422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:45.546320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 170/339] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:45.552227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac65e03-6c08-4a18-8e68-bdb79a3f20f5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:45.559506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a37722-7a7a-47e1-b32e-cae9c954aadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:45.560339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a37722-7a7a-47e1-b32e-cae9c954aadd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:45.560626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a37722-7a7a-47e1-b32e-cae9c954aadd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:45.560898 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a37722-7a7a-47e1-b32e-cae9c954aadd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:45.594520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a37722-7a7a-47e1-b32e-cae9c954aadd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-22a37722-7a7a-47e1-b32e-cae9c954aadd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:45.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:45.606677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:45.607481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:45.609299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 170/340] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:45.615915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a37722-7a7a-47e1-b32e-cae9c954aadd None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:45.735785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac65e03-6c08-4a18-8e68-bdb79a3f20f5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:45.762851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:45.780586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a37722-7a7a-47e1-b32e-cae9c954aadd None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:45.794624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429025cc-fa82-47c9-bb3e-9c4777aeba0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:45.795323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429025cc-fa82-47c9-bb3e-9c4777aeba0f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:45.795613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429025cc-fa82-47c9-bb3e-9c4777aeba0f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:45.795898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429025cc-fa82-47c9-bb3e-9c4777aeba0f None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:45.796667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429025cc-fa82-47c9-bb3e-9c4777aeba0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:45.796983 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429025cc-fa82-47c9-bb3e-9c4777aeba0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:45.798417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429025cc-fa82-47c9-bb3e-9c4777aeba0f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:45.808488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 171/341] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:45.813856 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429025cc-fa82-47c9-bb3e-9c4777aeba0f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:45.826619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edef2ce-0f3f-4ddb-a935-8791afe93b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:45.827314 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edef2ce-0f3f-4ddb-a935-8791afe93b4d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:45.827603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edef2ce-0f3f-4ddb-a935-8791afe93b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:45.827886 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edef2ce-0f3f-4ddb-a935-8791afe93b4d None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:45.828657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edef2ce-0f3f-4ddb-a935-8791afe93b4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:45.828969 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edef2ce-0f3f-4ddb-a935-8791afe93b4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:45.830410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edef2ce-0f3f-4ddb-a935-8791afe93b4d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:45.839589 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 171/342] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:45.844990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edef2ce-0f3f-4ddb-a935-8791afe93b4d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:45.873142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:45.968847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429025cc-fa82-47c9-bb3e-9c4777aeba0f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:46.021837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edef2ce-0f3f-4ddb-a935-8791afe93b4d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:46.028340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684082ba-a763-46b5-b86d-8c6a2d49a73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:46.028340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684082ba-a763-46b5-b86d-8c6a2d49a73b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:46.028340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684082ba-a763-46b5-b86d-8c6a2d49a73b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:46.028340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684082ba-a763-46b5-b86d-8c6a2d49a73b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:46.083165 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055ff8e-2484-4487-b9d0-493d41247472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:46.084681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055ff8e-2484-4487-b9d0-493d41247472 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:46.085176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055ff8e-2484-4487-b9d0-493d41247472 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:46.085695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055ff8e-2484-4487-b9d0-493d41247472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:46.104145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684082ba-a763-46b5-b86d-8c6a2d49a73b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:46.104145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684082ba-a763-46b5-b86d-8c6a2d49a73b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:46.116312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:46.154835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:46.212058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055ff8e-2484-4487-b9d0-493d41247472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:46.215486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055ff8e-2484-4487-b9d0-493d41247472 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:46.276684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 172/343] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:01:46.278549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826595c-bdf5-4ace-a2e4-21bd2ea87f3b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:46.280214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826595c-bdf5-4ace-a2e4-21bd2ea87f3b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:46.281403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826595c-bdf5-4ace-a2e4-21bd2ea87f3b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:46.283543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 173/344] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:46.286868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826595c-bdf5-4ace-a2e4-21bd2ea87f3b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:46.413683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 172/345] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:01:46.415400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daa891-7bab-421d-87ca-1f8dc9042a42 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:46.418309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daa891-7bab-421d-87ca-1f8dc9042a42 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:46.418800 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daa891-7bab-421d-87ca-1f8dc9042a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:46.450903 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-54daa891-7bab-421d-87ca-1f8dc9042a42 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:46.452205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-54daa891-7bab-421d-87ca-1f8dc9042a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:46.468785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826595c-bdf5-4ace-a2e4-21bd2ea87f3b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:46.509401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71555d48-8273-4bd7-8d31-71e9b9dce91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:46.510376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71555d48-8273-4bd7-8d31-71e9b9dce91b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:46.510659 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71555d48-8273-4bd7-8d31-71e9b9dce91b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:46.510933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71555d48-8273-4bd7-8d31-71e9b9dce91b None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:46.512873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71555d48-8273-4bd7-8d31-71e9b9dce91b None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:46.514507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71555d48-8273-4bd7-8d31-71e9b9dce91b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:46.516335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 174/346] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:46 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:46.517185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71555d48-8273-4bd7-8d31-71e9b9dce91b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:46.521363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:46.529270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:46.559749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 173/347] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:46.561683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50f91d-f7c7-4688-a1fe-59de55d679e6 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:46.562030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50f91d-f7c7-4688-a1fe-59de55d679e6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:46.562311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50f91d-f7c7-4688-a1fe-59de55d679e6 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:46.563935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 174/348] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:46.566953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50f91d-f7c7-4688-a1fe-59de55d679e6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:46.669513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71555d48-8273-4bd7-8d31-71e9b9dce91b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:46.713145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af5ed8-e92b-457f-956f-ecc4978fe567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:46.714658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af5ed8-e92b-457f-956f-ecc4978fe567 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:46.715094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50f91d-f7c7-4688-a1fe-59de55d679e6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:46.715324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af5ed8-e92b-457f-956f-ecc4978fe567 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:46.715778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af5ed8-e92b-457f-956f-ecc4978fe567 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:46.717131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af5ed8-e92b-457f-956f-ecc4978fe567 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:46.718796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af5ed8-e92b-457f-956f-ecc4978fe567 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:46.721034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 175/349] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:46 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:46.723616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973454e-cfb5-4ce5-a069-4387edaca073 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:46.724219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973454e-cfb5-4ce5-a069-4387edaca073 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:46.724684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973454e-cfb5-4ce5-a069-4387edaca073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:46.745369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f973454e-cfb5-4ce5-a069-4387edaca073 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:46.746855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f973454e-cfb5-4ce5-a069-4387edaca073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:46.759998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbe22e-5df1-4a12-937c-31c38c7c96d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:46.761657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbe22e-5df1-4a12-937c-31c38c7c96d1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:46.761657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbe22e-5df1-4a12-937c-31c38c7c96d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:46.761657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbe22e-5df1-4a12-937c-31c38c7c96d1 None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:46.769514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbe22e-5df1-4a12-937c-31c38c7c96d1 None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:46.771068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbe22e-5df1-4a12-937c-31c38c7c96d1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:46.773553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 175/350] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:46 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:46.774478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbe22e-5df1-4a12-937c-31c38c7c96d1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:46.829198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:46.839533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 176/351] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:46.840882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe21f93-d11e-43fb-a5c8-23d56c713a07 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:46.841226 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe21f93-d11e-43fb-a5c8-23d56c713a07 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:46.841456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe21f93-d11e-43fb-a5c8-23d56c713a07 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:46.843821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 177/352] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:46.844451 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe21f93-d11e-43fb-a5c8-23d56c713a07 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:46.847229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:46.906508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbe22e-5df1-4a12-937c-31c38c7c96d1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:46.944883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bfed1-44c3-4ef4-88ba-57e453fb9fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:46.945625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bfed1-44c3-4ef4-88ba-57e453fb9fbc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:46.945963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bfed1-44c3-4ef4-88ba-57e453fb9fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:46.946201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bfed1-44c3-4ef4-88ba-57e453fb9fbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:46.961420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bfed1-44c3-4ef4-88ba-57e453fb9fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d06bfed1-44c3-4ef4-88ba-57e453fb9fbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:46.969706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:46.970652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:46.972449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:46.974283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe21f93-d11e-43fb-a5c8-23d56c713a07 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:46.974785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 176/353] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:46.977194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bfed1-44c3-4ef4-88ba-57e453fb9fbc None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:47.016758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26288009-dc69-4297-9309-c608a171058c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:47.017629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26288009-dc69-4297-9309-c608a171058c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.017944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26288009-dc69-4297-9309-c608a171058c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.018224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26288009-dc69-4297-9309-c608a171058c None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.019404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26288009-dc69-4297-9309-c608a171058c None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:47.021061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26288009-dc69-4297-9309-c608a171058c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:47.022776 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 178/354] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:46 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:47.025067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1474b1e-e225-4786-ad52-6bffd61d79fc None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.025363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1474b1e-e225-4786-ad52-6bffd61d79fc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.025643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1474b1e-e225-4786-ad52-6bffd61d79fc None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.027162 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 179/355] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:47.030519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045091c5-e8c3-4431-8e35-4b6a43efad85 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.030519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045091c5-e8c3-4431-8e35-4b6a43efad85 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.030519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045091c5-e8c3-4431-8e35-4b6a43efad85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.048077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-045091c5-e8c3-4431-8e35-4b6a43efad85 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:47.049267 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-045091c5-e8c3-4431-8e35-4b6a43efad85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:47.098147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bfed1-44c3-4ef4-88ba-57e453fb9fbc None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:47.100130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:47.122771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:47.130947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 180/356] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:47.132518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045091c5-e8c3-4431-8e35-4b6a43efad85 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:47.142860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed998a-3fc6-419d-aefd-58b5d9b9f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:47.143534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed998a-3fc6-419d-aefd-58b5d9b9f821 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.143809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed998a-3fc6-419d-aefd-58b5d9b9f821 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.144394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed998a-3fc6-419d-aefd-58b5d9b9f821 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.145648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed998a-3fc6-419d-aefd-58b5d9b9f821 None admin] RBAC: Adding query filter params (group.id=caade8d95a1d41749bb1950641ad6d48, role.id=0b98291295724747acd01dd0d2306e62, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:47.145989 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed998a-3fc6-419d-aefd-58b5d9b9f821 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:47.148438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed998a-3fc6-419d-aefd-58b5d9b9f821 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:47.157120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 177/357] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 4 09:01:46 2022] GET /identity/v3/role_assignments?group.id=caade8d95a1d41749bb1950641ad6d48&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=0b98291295724747acd01dd0d2306e62 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:47.161871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed998a-3fc6-419d-aefd-58b5d9b9f821 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:47.269591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045091c5-e8c3-4431-8e35-4b6a43efad85 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:47.300963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed998a-3fc6-419d-aefd-58b5d9b9f821 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:47.312064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7ec18-c5d8-4aab-a31c-99c3b835c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:47.312996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7ec18-c5d8-4aab-a31c-99c3b835c9cf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.313283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7ec18-c5d8-4aab-a31c-99c3b835c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.313560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7ec18-c5d8-4aab-a31c-99c3b835c9cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.327875 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7ec18-c5d8-4aab-a31c-99c3b835c9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-eeb7ec18-c5d8-4aab-a31c-99c3b835c9cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:47.337064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:47.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 181/358] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:47.338903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7ec18-c5d8-4aab-a31c-99c3b835c9cf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:47.348717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b727a0-4653-4146-b5e5-4ffc4d855501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:47.349645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b727a0-4653-4146-b5e5-4ffc4d855501 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.349951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b727a0-4653-4146-b5e5-4ffc4d855501 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.350224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b727a0-4653-4146-b5e5-4ffc4d855501 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.351090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b727a0-4653-4146-b5e5-4ffc4d855501 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:47.351437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b727a0-4653-4146-b5e5-4ffc4d855501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:47.352945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b727a0-4653-4146-b5e5-4ffc4d855501 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:47.364221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 178/359] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:47.366646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac3cbc-f4de-4b55-8aca-2463f6140b64 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.366930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac3cbc-f4de-4b55-8aca-2463f6140b64 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.367203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac3cbc-f4de-4b55-8aca-2463f6140b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.388079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-49ac3cbc-f4de-4b55-8aca-2463f6140b64 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:47.389258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-49ac3cbc-f4de-4b55-8aca-2463f6140b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:47.406307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:47.413951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:47.460587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7ec18-c5d8-4aab-a31c-99c3b835c9cf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:47.474946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 179/360] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:47.475242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac3cbc-f4de-4b55-8aca-2463f6140b64 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:47.510773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad62300-60ee-404e-bfb6-9fc7b7c8e67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:47.510978 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad62300-60ee-404e-bfb6-9fc7b7c8e67f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.511219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad62300-60ee-404e-bfb6-9fc7b7c8e67f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.511421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad62300-60ee-404e-bfb6-9fc7b7c8e67f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.512270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad62300-60ee-404e-bfb6-9fc7b7c8e67f None admin] RBAC: Adding query filter params (group.id=caade8d95a1d41749bb1950641ad6d48, role.id=2eea147ab8b1426294a08106410f6d45, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:47.512584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad62300-60ee-404e-bfb6-9fc7b7c8e67f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:47.514085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad62300-60ee-404e-bfb6-9fc7b7c8e67f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:47.526150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 182/361] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 4 09:01:47 2022] GET /identity/v3/role_assignments?group.id=caade8d95a1d41749bb1950641ad6d48&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=2eea147ab8b1426294a08106410f6d45 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:47.527331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad62300-60ee-404e-bfb6-9fc7b7c8e67f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:47.619739 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac3cbc-f4de-4b55-8aca-2463f6140b64 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:47.673824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf9517-9d49-428a-aa8a-1eac9891c06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:47.674692 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf9517-9d49-428a-aa8a-1eac9891c06a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.674977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf9517-9d49-428a-aa8a-1eac9891c06a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.675255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf9517-9d49-428a-aa8a-1eac9891c06a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.676121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad62300-60ee-404e-bfb6-9fc7b7c8e67f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:47.690823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf9517-9d49-428a-aa8a-1eac9891c06a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-24cf9517-9d49-428a-aa8a-1eac9891c06a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:47.698907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:47.699965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:47.700813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 180/362] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:47.701507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf9517-9d49-428a-aa8a-1eac9891c06a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:47.710285 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:47.724244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed882e8-4d5c-4775-84ba-14e6fa396c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:47.725235 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed882e8-4d5c-4775-84ba-14e6fa396c57 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.725666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed882e8-4d5c-4775-84ba-14e6fa396c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.726201 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed882e8-4d5c-4775-84ba-14e6fa396c57 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.727129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed882e8-4d5c-4775-84ba-14e6fa396c57 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:47.727585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed882e8-4d5c-4775-84ba-14e6fa396c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:47.729126 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed882e8-4d5c-4775-84ba-14e6fa396c57 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:47.733348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:47.742079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 183/363] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:47.742867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed882e8-4d5c-4775-84ba-14e6fa396c57 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:47.832642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf9517-9d49-428a-aa8a-1eac9891c06a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:47.869591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9a802-8d65-4ecd-9711-456100baeead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:47.870370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9a802-8d65-4ecd-9711-456100baeead None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.870661 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9a802-8d65-4ecd-9711-456100baeead None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.871049 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9a802-8d65-4ecd-9711-456100baeead None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.881954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed882e8-4d5c-4775-84ba-14e6fa396c57 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:47.929941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901afec0-610e-4a15-a903-cdfd610e65bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:47.930804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901afec0-610e-4a15-a903-cdfd610e65bb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:47.931227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901afec0-610e-4a15-a903-cdfd610e65bb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:47.931636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901afec0-610e-4a15-a903-cdfd610e65bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:47.954427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901afec0-610e-4a15-a903-cdfd610e65bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-901afec0-610e-4a15-a903-cdfd610e65bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:47.963364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:47.964340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:47.965150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 184/364] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:01:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:47.965871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901afec0-610e-4a15-a903-cdfd610e65bb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:47.977260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9a802-8d65-4ecd-9711-456100baeead None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:47.981830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9a802-8d65-4ecd-9711-456100baeead None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:48.007786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:48.025245 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:48.088284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901afec0-610e-4a15-a903-cdfd610e65bb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:48.107103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 181/365] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:48.109496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9a802-8d65-4ecd-9711-456100baeead None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:48.143764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28dbb3-8dd8-4396-a54a-a4ae0aa8d90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:48.145177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28dbb3-8dd8-4396-a54a-a4ae0aa8d90f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:48.145714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28dbb3-8dd8-4396-a54a-a4ae0aa8d90f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:48.146211 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28dbb3-8dd8-4396-a54a-a4ae0aa8d90f None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:48.148935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28dbb3-8dd8-4396-a54a-a4ae0aa8d90f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:48.149460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28dbb3-8dd8-4396-a54a-a4ae0aa8d90f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:48.152556 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28dbb3-8dd8-4396-a54a-a4ae0aa8d90f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:48.165088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 185/366] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:48.167742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28dbb3-8dd8-4396-a54a-a4ae0aa8d90f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:48.228795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9a802-8d65-4ecd-9711-456100baeead None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:48.247733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:48.267123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f473f0-91e1-46b5-bed3-b873308526a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:48.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f473f0-91e1-46b5-bed3-b873308526a0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:48.268261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f473f0-91e1-46b5-bed3-b873308526a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:48.268543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f473f0-91e1-46b5-bed3-b873308526a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:48.279105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28dbb3-8dd8-4396-a54a-a4ae0aa8d90f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:48.290272 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:48.316112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c83ac-48f3-4104-bf6d-4ae86ce7857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:48.316827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c83ac-48f3-4104-bf6d-4ae86ce7857b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:48.317113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c83ac-48f3-4104-bf6d-4ae86ce7857b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:48.317398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c83ac-48f3-4104-bf6d-4ae86ce7857b None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:48.318191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c83ac-48f3-4104-bf6d-4ae86ce7857b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:48.318598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c83ac-48f3-4104-bf6d-4ae86ce7857b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:48.320092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c83ac-48f3-4104-bf6d-4ae86ce7857b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:48.330590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 186/367] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:48.331348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c83ac-48f3-4104-bf6d-4ae86ce7857b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:48.344465 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f473f0-91e1-46b5-bed3-b873308526a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:48.346014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f473f0-91e1-46b5-bed3-b873308526a0 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:48.445776 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c83ac-48f3-4104-bf6d-4ae86ce7857b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:48.455627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 182/368] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:48.456560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f473f0-91e1-46b5-bed3-b873308526a0 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:48.482253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ade6a4-a281-41aa-9b7b-4454a1158f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:48.482489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:48.483130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ade6a4-a281-41aa-9b7b-4454a1158f5d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:48.483414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ade6a4-a281-41aa-9b7b-4454a1158f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:48.483690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ade6a4-a281-41aa-9b7b-4454a1158f5d None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:48.484896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ade6a4-a281-41aa-9b7b-4454a1158f5d None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:48.486382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ade6a4-a281-41aa-9b7b-4454a1158f5d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:48.488091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 187/369] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:48 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:48.488901 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ade6a4-a281-41aa-9b7b-4454a1158f5d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:48.544440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:48.568611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f473f0-91e1-46b5-bed3-b873308526a0 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:48.599971 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ade6a4-a281-41aa-9b7b-4454a1158f5d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:48.603459 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6391b0e-aa25-4e32-8273-7de3e31b7000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:48.604346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391b0e-aa25-4e32-8273-7de3e31b7000 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:48.604631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391b0e-aa25-4e32-8273-7de3e31b7000 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:48.604906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391b0e-aa25-4e32-8273-7de3e31b7000 None admin] PATH_INFO: `/v3/users/866b9a3a2e2b4575adb018eeedf06285` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:48.606141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6391b0e-aa25-4e32-8273-7de3e31b7000 None admin] RBAC: Authorizing `identity:get_user(user_id=866b9a3a2e2b4575adb018eeedf06285)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:48.607754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6391b0e-aa25-4e32-8273-7de3e31b7000 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:48.609754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 183/370] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:01:48 2022] GET /identity/v3/users/866b9a3a2e2b4575adb018eeedf06285 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:48.610709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6391b0e-aa25-4e32-8273-7de3e31b7000 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:48.638447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ffe39-748b-42b6-a3cb-99398a1b706a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:48.639110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ffe39-748b-42b6-a3cb-99398a1b706a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:48.639394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ffe39-748b-42b6-a3cb-99398a1b706a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:48.639671 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ffe39-748b-42b6-a3cb-99398a1b706a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:48.704468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:48.713805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ffe39-748b-42b6-a3cb-99398a1b706a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:48.715457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ffe39-748b-42b6-a3cb-99398a1b706a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:48.721601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6391b0e-aa25-4e32-8273-7de3e31b7000 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:48.759274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8bdc7-c733-497a-a619-45cb389c3662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:48.760146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8bdc7-c733-497a-a619-45cb389c3662 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:48.760431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8bdc7-c733-497a-a619-45cb389c3662 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:48.760708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8bdc7-c733-497a-a619-45cb389c3662 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:48.761950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8bdc7-c733-497a-a619-45cb389c3662 None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:48.762220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:48.763421 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8bdc7-c733-497a-a619-45cb389c3662 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:48.765147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 184/371] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:48 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:48.766059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8bdc7-c733-497a-a619-45cb389c3662 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:48.828438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 188/372] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:48.829313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ffe39-748b-42b6-a3cb-99398a1b706a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:48.879187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8bdc7-c733-497a-a619-45cb389c3662 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:48.918052 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3758c42-e33d-4332-8317-cc5e166f3c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:48.918929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3758c42-e33d-4332-8317-cc5e166f3c41 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:48.919214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3758c42-e33d-4332-8317-cc5e166f3c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:48.919499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3758c42-e33d-4332-8317-cc5e166f3c41 None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:48.920697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3758c42-e33d-4332-8317-cc5e166f3c41 None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:48.922192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3758c42-e33d-4332-8317-cc5e166f3c41 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:48.923947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 185/373] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:48 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:48.929008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3758c42-e33d-4332-8317-cc5e166f3c41 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:48.936856 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ffe39-748b-42b6-a3cb-99398a1b706a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:48.958186 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:48.978763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7aaf3-8916-46cb-9330-6affff897dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:48.979678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7aaf3-8916-46cb-9330-6affff897dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:48.980103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7aaf3-8916-46cb-9330-6affff897dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:48.980518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7aaf3-8916-46cb-9330-6affff897dd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:48.981607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7aaf3-8916-46cb-9330-6affff897dd1 None admin] RBAC: Adding query filter params (role.id=67473bdffb674c6b8d0afd413642eff6, user.id=866b9a3a2e2b4575adb018eeedf06285, scope.system=all) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:48.982148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7aaf3-8916-46cb-9330-6affff897dd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:48.983756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7aaf3-8916-46cb-9330-6affff897dd1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:48.994218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:48.998274 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 189/374] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 4 09:01:48 2022] GET /identity/v3/role_assignments?user.id=866b9a3a2e2b4575adb018eeedf06285&scope.system=all&role.id=67473bdffb674c6b8d0afd413642eff6 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:48.999185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7aaf3-8916-46cb-9330-6affff897dd1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:49.043225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3758c42-e33d-4332-8317-cc5e166f3c41 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:49.082293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd1161-d79c-4363-a5bb-24821604abb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:49.083000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd1161-d79c-4363-a5bb-24821604abb2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:49.083288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd1161-d79c-4363-a5bb-24821604abb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:49.083563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd1161-d79c-4363-a5bb-24821604abb2 None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:49.084792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd1161-d79c-4363-a5bb-24821604abb2 None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:49.086320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd1161-d79c-4363-a5bb-24821604abb2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:49.087976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 186/375] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:48 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:49.088843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd1161-d79c-4363-a5bb-24821604abb2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:49.115882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7aaf3-8916-46cb-9330-6affff897dd1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:49.160612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64904536-ecc3-4725-89e5-e31a1669f6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:49.160612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64904536-ecc3-4725-89e5-e31a1669f6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:49.160612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64904536-ecc3-4725-89e5-e31a1669f6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:49.160612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64904536-ecc3-4725-89e5-e31a1669f6e6 None admin] PATH_INFO: `/v3/projects/3f0a53c420a84618ab851430c484608a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:49.161441 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64904536-ecc3-4725-89e5-e31a1669f6e6 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0a53c420a84618ab851430c484608a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:49.161441 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64904536-ecc3-4725-89e5-e31a1669f6e6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:49.163886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 190/376] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:48 2022] GET /identity/v3/projects/3f0a53c420a84618ab851430c484608a => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:49.163886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64904536-ecc3-4725-89e5-e31a1669f6e6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:49.196453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:49.208419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd1161-d79c-4363-a5bb-24821604abb2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:49.242330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:49.250002 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf8ac2-baa0-4df3-9f6d-e6d3cffb80ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:49.250998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf8ac2-baa0-4df3-9f6d-e6d3cffb80ef None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:49.251529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf8ac2-baa0-4df3-9f6d-e6d3cffb80ef None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:49.252331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf8ac2-baa0-4df3-9f6d-e6d3cffb80ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:49.253386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf8ac2-baa0-4df3-9f6d-e6d3cffb80ef None admin] RBAC: Adding query filter params (group.id=caade8d95a1d41749bb1950641ad6d48, role.id=0b98291295724747acd01dd0d2306e62, scope.project.id=7e689a8addd941f8a2efd52508e178b7) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:49.254005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf8ac2-baa0-4df3-9f6d-e6d3cffb80ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:49.255733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf8ac2-baa0-4df3-9f6d-e6d3cffb80ef None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:49.262696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 187/377] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 4 09:01:49 2022] GET /identity/v3/role_assignments?group.id=caade8d95a1d41749bb1950641ad6d48&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=0b98291295724747acd01dd0d2306e62 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:49.264122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf8ac2-baa0-4df3-9f6d-e6d3cffb80ef None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:49.308692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64904536-ecc3-4725-89e5-e31a1669f6e6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:49.364588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ef536-feb9-4f51-a17b-be2a4389e621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:49.365271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ef536-feb9-4f51-a17b-be2a4389e621 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:49.365560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ef536-feb9-4f51-a17b-be2a4389e621 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:49.365850 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ef536-feb9-4f51-a17b-be2a4389e621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:49.366723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ef536-feb9-4f51-a17b-be2a4389e621 None admin] RBAC: Adding query filter params (group.id=caade8d95a1d41749bb1950641ad6d48, role.id=2eea147ab8b1426294a08106410f6d45, scope.project.id=7e689a8addd941f8a2efd52508e178b7) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:49.367036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ef536-feb9-4f51-a17b-be2a4389e621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:49.368491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ef536-feb9-4f51-a17b-be2a4389e621 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:49.386078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf8ac2-baa0-4df3-9f6d-e6d3cffb80ef None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:49.388323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 191/378] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 4 09:01:49 2022] GET /identity/v3/role_assignments?group.id=caade8d95a1d41749bb1950641ad6d48&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=2eea147ab8b1426294a08106410f6d45 => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:49.427755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c02c69-5bc3-4899-8bf9-84f39d295dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:49.428946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c02c69-5bc3-4899-8bf9-84f39d295dfb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:49.429422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c02c69-5bc3-4899-8bf9-84f39d295dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:49.429907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c02c69-5bc3-4899-8bf9-84f39d295dfb None admin] PATH_INFO: `/v3/groups/26a9090c30004262a57805eed2b95902` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:49.436588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c02c69-5bc3-4899-8bf9-84f39d295dfb None admin] RBAC: Authorizing `identity:get_group(group_id=26a9090c30004262a57805eed2b95902)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:49.438387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c02c69-5bc3-4899-8bf9-84f39d295dfb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:49.440441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 188/379] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:49 2022] GET /identity/v3/groups/26a9090c30004262a57805eed2b95902 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:49.446930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ef536-feb9-4f51-a17b-be2a4389e621 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:49.569522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ef536-feb9-4f51-a17b-be2a4389e621 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:49.598716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ea946-2b1a-443a-a019-3ead1ec81b6f None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:49.599003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ea946-2b1a-443a-a019-3ead1ec81b6f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:49.599286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ea946-2b1a-443a-a019-3ead1ec81b6f None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:49.600935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 189/380] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:49.604207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:49.607525 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea4ca0-e68a-4085-9c8b-ec9dda579aab None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:49.607845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea4ca0-e68a-4085-9c8b-ec9dda579aab None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:49.608107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea4ca0-e68a-4085-9c8b-ec9dda579aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:49.616197 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9414a7-7713-4092-bbcf-c955929630c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:49.618144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9414a7-7713-4092-bbcf-c955929630c8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:49.618647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9414a7-7713-4092-bbcf-c955929630c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:49.619098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9414a7-7713-4092-bbcf-c955929630c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:49.620145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9414a7-7713-4092-bbcf-c955929630c8 None admin] RBAC: Adding query filter params (group.id=26a9090c30004262a57805eed2b95902, role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=3f0a53c420a84618ab851430c484608a) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:49.620708 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9414a7-7713-4092-bbcf-c955929630c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:49.622435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9414a7-7713-4092-bbcf-c955929630c8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:49.628560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8eea4ca0-e68a-4085-9c8b-ec9dda579aab None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:49.629962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8eea4ca0-e68a-4085-9c8b-ec9dda579aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:49.631341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 192/381] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 4 09:01:49 2022] GET /identity/v3/role_assignments?group.id=26a9090c30004262a57805eed2b95902&scope.project.id=3f0a53c420a84618ab851430c484608a&role.id=2302517080ac42578aa81c606dd0ff2f => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:49.674869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:49.716359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 190/382] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:49.742745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9414a7-7713-4092-bbcf-c955929630c8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:49.869020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9414a7-7713-4092-bbcf-c955929630c8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:49.906157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a111ca-642c-4ba6-80bf-fb6a3b8285de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:49.906975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a111ca-642c-4ba6-80bf-fb6a3b8285de None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:49.907259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a111ca-642c-4ba6-80bf-fb6a3b8285de None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:49.907533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a111ca-642c-4ba6-80bf-fb6a3b8285de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:49.918880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a111ca-642c-4ba6-80bf-fb6a3b8285de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-83a111ca-642c-4ba6-80bf-fb6a3b8285de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:49.926552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:49.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:49.928261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 193/383] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:49.933975 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea4ca0-e68a-4085-9c8b-ec9dda579aab None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:50.054446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea4ca0-e68a-4085-9c8b-ec9dda579aab None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:50.092892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bdd7f-7298-4def-9e52-9bf565d00018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:50.093556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bdd7f-7298-4def-9e52-9bf565d00018 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:50.093837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bdd7f-7298-4def-9e52-9bf565d00018 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:50.094145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bdd7f-7298-4def-9e52-9bf565d00018 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:50.094905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bdd7f-7298-4def-9e52-9bf565d00018 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:50.095208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bdd7f-7298-4def-9e52-9bf565d00018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:50.096578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bdd7f-7298-4def-9e52-9bf565d00018 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:50.107660 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 191/384] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:50.113611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a111ca-642c-4ba6-80bf-fb6a3b8285de None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:50.144136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed292ed-e23e-447f-bbb7-3574c02f75a5 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:50.144502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed292ed-e23e-447f-bbb7-3574c02f75a5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:50.145228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed292ed-e23e-447f-bbb7-3574c02f75a5 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:50.147068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 192/385] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:50.158763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44458fa1-93f2-4897-bb94-4a7f657a9c6f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:50.159297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44458fa1-93f2-4897-bb94-4a7f657a9c6f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:50.159907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44458fa1-93f2-4897-bb94-4a7f657a9c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:50.181073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-44458fa1-93f2-4897-bb94-4a7f657a9c6f None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:50.182623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-44458fa1-93f2-4897-bb94-4a7f657a9c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:50.186583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:50.206513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:50.277554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 193/386] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:50.305345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a111ca-642c-4ba6-80bf-fb6a3b8285de None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:50.335351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44458fa1-93f2-4897-bb94-4a7f657a9c6f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:50.349250 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4012e-611b-4cfc-92ad-02f59bb2c0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:50.352380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4012e-611b-4cfc-92ad-02f59bb2c0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:50.352380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4012e-611b-4cfc-92ad-02f59bb2c0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:50.352380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4012e-611b-4cfc-92ad-02f59bb2c0d7 None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:50.352380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4012e-611b-4cfc-92ad-02f59bb2c0d7 None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:50.353425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4012e-611b-4cfc-92ad-02f59bb2c0d7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:50.355312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 194/387] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:50 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:50.368203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4012e-611b-4cfc-92ad-02f59bb2c0d7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:50.496337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4012e-611b-4cfc-92ad-02f59bb2c0d7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:50.512670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44458fa1-93f2-4897-bb94-4a7f657a9c6f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:50.517687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:50.542044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ee67e-5b53-4fe4-9993-791b25ab3fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:50.543255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ee67e-5b53-4fe4-9993-791b25ab3fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:50.543750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ee67e-5b53-4fe4-9993-791b25ab3fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:50.544230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ee67e-5b53-4fe4-9993-791b25ab3fc9 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:50.550682 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ee67e-5b53-4fe4-9993-791b25ab3fc9 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:50.551205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef228c64-b9fb-45c6-89b1-3a19fb5273c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:50.552026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef228c64-b9fb-45c6-89b1-3a19fb5273c6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:50.552307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef228c64-b9fb-45c6-89b1-3a19fb5273c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:50.552587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef228c64-b9fb-45c6-89b1-3a19fb5273c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:50.553435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ee67e-5b53-4fe4-9993-791b25ab3fc9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:50.555901 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 195/388] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:50 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:50.561689 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ee67e-5b53-4fe4-9993-791b25ab3fc9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:50.563670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef228c64-b9fb-45c6-89b1-3a19fb5273c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ef228c64-b9fb-45c6-89b1-3a19fb5273c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:50.572044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:50.572995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:50.573838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 194/389] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:01:50.574586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:50.583047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef228c64-b9fb-45c6-89b1-3a19fb5273c6 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:50.688541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ee67e-5b53-4fe4-9993-791b25ab3fc9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:50.734549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd210a-1936-443d-8c79-508704138c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:50.735831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd210a-1936-443d-8c79-508704138c42 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:50.736333 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd210a-1936-443d-8c79-508704138c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:50.736796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd210a-1936-443d-8c79-508704138c42 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/groups/caade8d95a1d41749bb1950641ad6d48/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:50.739336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd210a-1936-443d-8c79-508704138c42 None admin] RBAC: Authorizing `identity:create_grant(project_id=795c9a5eff6345f6ae2de8e24f361e51, group_id=caade8d95a1d41749bb1950641ad6d48, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:50.741238 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd210a-1936-443d-8c79-508704138c42 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:50.758033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 196/390] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 4 09:01:50 2022] PUT /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/groups/caade8d95a1d41749bb1950641ad6d48/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:01:50.790437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef228c64-b9fb-45c6-89b1-3a19fb5273c6 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:50.851510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2aa44-4191-45a2-a256-c6a8c48e64a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:50.852210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2aa44-4191-45a2-a256-c6a8c48e64a9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:50.852493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2aa44-4191-45a2-a256-c6a8c48e64a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:50.852769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2aa44-4191-45a2-a256-c6a8c48e64a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:50.853550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2aa44-4191-45a2-a256-c6a8c48e64a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:50.853859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2aa44-4191-45a2-a256-c6a8c48e64a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:50.855309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2aa44-4191-45a2-a256-c6a8c48e64a9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:50.864845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 195/391] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:50.870555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd210a-1936-443d-8c79-508704138c42 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:50.960247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:51.019874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd210a-1936-443d-8c79-508704138c42 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:51.058077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3c85c-d023-4cec-b7ba-2d264a52a3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:51.058956 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3c85c-d023-4cec-b7ba-2d264a52a3fa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:51.059242 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3c85c-d023-4cec-b7ba-2d264a52a3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:51.059521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3c85c-d023-4cec-b7ba-2d264a52a3fa None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:51.060707 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3c85c-d023-4cec-b7ba-2d264a52a3fa None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:51.062190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3c85c-d023-4cec-b7ba-2d264a52a3fa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:51.065168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 197/392] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:50 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:51.070623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2aa44-4191-45a2-a256-c6a8c48e64a9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:51.199540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:51.252947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2aa44-4191-45a2-a256-c6a8c48e64a9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:51.301032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9dc72-75fe-41a8-9791-ad3c258bd2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:51.301962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9dc72-75fe-41a8-9791-ad3c258bd2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:51.302255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9dc72-75fe-41a8-9791-ad3c258bd2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:51.302528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9dc72-75fe-41a8-9791-ad3c258bd2a3 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:51.306761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9dc72-75fe-41a8-9791-ad3c258bd2a3 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:51.308234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9dc72-75fe-41a8-9791-ad3c258bd2a3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:51.310364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 196/393] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:51 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:51.316289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3c85c-d023-4cec-b7ba-2d264a52a3fa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:51.439953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3c85c-d023-4cec-b7ba-2d264a52a3fa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:51.456467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:51.474837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487aabb4-2de6-4119-8bb1-99e51c54f153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:51.475732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487aabb4-2de6-4119-8bb1-99e51c54f153 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:51.476015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487aabb4-2de6-4119-8bb1-99e51c54f153 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:51.476290 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487aabb4-2de6-4119-8bb1-99e51c54f153 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/groups/caade8d95a1d41749bb1950641ad6d48/roles/2eea147ab8b1426294a08106410f6d45` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:51.478563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487aabb4-2de6-4119-8bb1-99e51c54f153 None admin] RBAC: Authorizing `identity:create_grant(project_id=795c9a5eff6345f6ae2de8e24f361e51, group_id=caade8d95a1d41749bb1950641ad6d48, role_id=2eea147ab8b1426294a08106410f6d45)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:51.480009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487aabb4-2de6-4119-8bb1-99e51c54f153 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:51.502159 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 198/394] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 4 09:01:51 2022] PUT /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51/groups/caade8d95a1d41749bb1950641ad6d48/roles/2eea147ab8b1426294a08106410f6d45 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:52.029645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14fcf2-5559-4d37-a46a-0dfd7e6bff47 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.030664 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14fcf2-5559-4d37-a46a-0dfd7e6bff47 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.031147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14fcf2-5559-4d37-a46a-0dfd7e6bff47 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.032974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 197/395] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:52.038705 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716e94a-f10f-4677-bf3c-d5f9dc17105e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.038996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716e94a-f10f-4677-bf3c-d5f9dc17105e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.039276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716e94a-f10f-4677-bf3c-d5f9dc17105e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.060084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9716e94a-f10f-4677-bf3c-d5f9dc17105e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:52.061276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9716e94a-f10f-4677-bf3c-d5f9dc17105e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:52.156266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 199/396] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:52.190913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14fcf2-5559-4d37-a46a-0dfd7e6bff47 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:52.268891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06627874-6fcf-427a-a0e2-478be59a38f5 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.269213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06627874-6fcf-427a-a0e2-478be59a38f5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.269766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06627874-6fcf-427a-a0e2-478be59a38f5 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.272841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 200/397] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:52.277824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06c33a-33af-4ca3-ac55-b7f78b31ca89 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.278145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06c33a-33af-4ca3-ac55-b7f78b31ca89 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.278423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06c33a-33af-4ca3-ac55-b7f78b31ca89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.305315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9e06c33a-33af-4ca3-ac55-b7f78b31ca89 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:52.312240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9e06c33a-33af-4ca3-ac55-b7f78b31ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:52.355863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14fcf2-5559-4d37-a46a-0dfd7e6bff47 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:52.389668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:52.398730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d343d-e58b-4938-82b2-b7b7fb94b202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:52.399953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d343d-e58b-4938-82b2-b7b7fb94b202 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.400446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d343d-e58b-4938-82b2-b7b7fb94b202 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.400937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d343d-e58b-4938-82b2-b7b7fb94b202 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.404198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 201/398] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:52.410127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57775aa2-eb71-4b13-a5ca-7b895aa23e5b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.410127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57775aa2-eb71-4b13-a5ca-7b895aa23e5b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.410127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57775aa2-eb71-4b13-a5ca-7b895aa23e5b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.410127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 202/399] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:52.412943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a2365-41a6-4d5f-881c-c68364539bfd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.413232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a2365-41a6-4d5f-881c-c68364539bfd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.413513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a2365-41a6-4d5f-881c-c68364539bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.420938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d343d-e58b-4938-82b2-b7b7fb94b202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:52.438286 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-465a2365-41a6-4d5f-881c-c68364539bfd None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:52.438286 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-465a2365-41a6-4d5f-881c-c68364539bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-277d343d-e58b-4938-82b2-b7b7fb94b202 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:52.439676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:52.440597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:52.441871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:52.444764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 198/400] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:52.450840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d343d-e58b-4938-82b2-b7b7fb94b202 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:52.470257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:52.522002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 203/401] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:52.523029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a2365-41a6-4d5f-881c-c68364539bfd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:52.578700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d343d-e58b-4938-82b2-b7b7fb94b202 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:52.616562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70042e3f-37a5-4c19-a96b-35e68f56cf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:52.617622 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70042e3f-37a5-4c19-a96b-35e68f56cf48 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.618139 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70042e3f-37a5-4c19-a96b-35e68f56cf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.618615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70042e3f-37a5-4c19-a96b-35e68f56cf48 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.619671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70042e3f-37a5-4c19-a96b-35e68f56cf48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:52.620293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70042e3f-37a5-4c19-a96b-35e68f56cf48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:52.621909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70042e3f-37a5-4c19-a96b-35e68f56cf48 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:52.632816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 199/402] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:52.639006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70042e3f-37a5-4c19-a96b-35e68f56cf48 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:52.643746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a2365-41a6-4d5f-881c-c68364539bfd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:52.680449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484cd4c-6ca6-4a99-b0fa-41574620617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:52.680996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484cd4c-6ca6-4a99-b0fa-41574620617e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.681284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484cd4c-6ca6-4a99-b0fa-41574620617e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.681559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484cd4c-6ca6-4a99-b0fa-41574620617e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.692762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c484cd4c-6ca6-4a99-b0fa-41574620617e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:52.700885 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c484cd4c-6ca6-4a99-b0fa-41574620617e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:52.701771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:52.703317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:52.704188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:52.706763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 204/403] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:01:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:52.707993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484cd4c-6ca6-4a99-b0fa-41574620617e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:52.741802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:52.764611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70042e3f-37a5-4c19-a96b-35e68f56cf48 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:52.802771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec821d6b-7f45-4f1a-8d42-ae9a06b2491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:52.803541 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec821d6b-7f45-4f1a-8d42-ae9a06b2491b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.803829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec821d6b-7f45-4f1a-8d42-ae9a06b2491b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.804105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec821d6b-7f45-4f1a-8d42-ae9a06b2491b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.815318 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec821d6b-7f45-4f1a-8d42-ae9a06b2491b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:52.822120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484cd4c-6ca6-4a99-b0fa-41574620617e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ec821d6b-7f45-4f1a-8d42-ae9a06b2491b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:52.824389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:52.825279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:52.826102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:52.830287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 200/404] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:52.831104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec821d6b-7f45-4f1a-8d42-ae9a06b2491b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:52.863577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f7f81-5002-45bb-85b9-cf2277deb601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:52.864907 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f7f81-5002-45bb-85b9-cf2277deb601 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.865455 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f7f81-5002-45bb-85b9-cf2277deb601 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.866040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f7f81-5002-45bb-85b9-cf2277deb601 None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.867742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f7f81-5002-45bb-85b9-cf2277deb601 None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:52.869573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f7f81-5002-45bb-85b9-cf2277deb601 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:52.871588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 205/405] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:52 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:52.872660 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f7f81-5002-45bb-85b9-cf2277deb601 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:52.955246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec821d6b-7f45-4f1a-8d42-ae9a06b2491b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:52.969501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:52.981663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:52.989503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f7f81-5002-45bb-85b9-cf2277deb601 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:52.996484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9dc81-65b0-4c15-9a35-66506b68039b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:52.997164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9dc81-65b0-4c15-9a35-66506b68039b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:52.997452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9dc81-65b0-4c15-9a35-66506b68039b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:52.997735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9dc81-65b0-4c15-9a35-66506b68039b None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:52.998641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9dc81-65b0-4c15-9a35-66506b68039b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:52.998958 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9dc81-65b0-4c15-9a35-66506b68039b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:53.000362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9dc81-65b0-4c15-9a35-66506b68039b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:53.015486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 201/406] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:53.016482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9dc81-65b0-4c15-9a35-66506b68039b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:53.028618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deef574-bca3-48af-bca8-55ed27d461d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:53.029744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deef574-bca3-48af-bca8-55ed27d461d4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.030269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deef574-bca3-48af-bca8-55ed27d461d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.030832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deef574-bca3-48af-bca8-55ed27d461d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.031971 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deef574-bca3-48af-bca8-55ed27d461d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:53.032672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deef574-bca3-48af-bca8-55ed27d461d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:53.034590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deef574-bca3-48af-bca8-55ed27d461d4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:53.044090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 206/407] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:53.045384 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deef574-bca3-48af-bca8-55ed27d461d4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:53.132054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9dc81-65b0-4c15-9a35-66506b68039b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:53.156060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deef574-bca3-48af-bca8-55ed27d461d4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:53.171838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aaadcc-0b78-48bf-97c6-8cbfa9ebc0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:53.172717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaadcc-0b78-48bf-97c6-8cbfa9ebc0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.173004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaadcc-0b78-48bf-97c6-8cbfa9ebc0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.173285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaadcc-0b78-48bf-97c6-8cbfa9ebc0e1 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.174511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aaadcc-0b78-48bf-97c6-8cbfa9ebc0e1 None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:53.176823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aaadcc-0b78-48bf-97c6-8cbfa9ebc0e1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:53.178773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 202/408] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:53 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:53.179353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aaadcc-0b78-48bf-97c6-8cbfa9ebc0e1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:53.195044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cdb0d-dbc0-49e0-8b7c-733018eabb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:53.196358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cdb0d-dbc0-49e0-8b7c-733018eabb7c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.196890 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cdb0d-dbc0-49e0-8b7c-733018eabb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.197367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cdb0d-dbc0-49e0-8b7c-733018eabb7c None admin] PATH_INFO: `/v3/groups/26a9090c30004262a57805eed2b95902` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.198924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659cdb0d-dbc0-49e0-8b7c-733018eabb7c None admin] RBAC: Authorizing `identity:get_group(group_id=26a9090c30004262a57805eed2b95902)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:53.200843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659cdb0d-dbc0-49e0-8b7c-733018eabb7c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:53.203157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 207/409] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:53 2022] GET /identity/v3/groups/26a9090c30004262a57805eed2b95902 => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:53.204236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cdb0d-dbc0-49e0-8b7c-733018eabb7c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:53.236949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:53.238269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:53.299009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aaadcc-0b78-48bf-97c6-8cbfa9ebc0e1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:53.319790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cdb0d-dbc0-49e0-8b7c-733018eabb7c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:53.335887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f067396-3b7a-4251-b2d2-c3eb08401f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:53.336754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f067396-3b7a-4251-b2d2-c3eb08401f53 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.337037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f067396-3b7a-4251-b2d2-c3eb08401f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.337298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f067396-3b7a-4251-b2d2-c3eb08401f53 None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.338507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f067396-3b7a-4251-b2d2-c3eb08401f53 None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:53.339945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f067396-3b7a-4251-b2d2-c3eb08401f53 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:53.341579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 203/410] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:53 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:53.346538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f067396-3b7a-4251-b2d2-c3eb08401f53 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:53.363602 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159be622-e6f4-42a8-b819-2147e4a35002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:53.364876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159be622-e6f4-42a8-b819-2147e4a35002 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.365169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159be622-e6f4-42a8-b819-2147e4a35002 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.365444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159be622-e6f4-42a8-b819-2147e4a35002 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7/groups/caade8d95a1d41749bb1950641ad6d48/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.367596 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159be622-e6f4-42a8-b819-2147e4a35002 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e689a8addd941f8a2efd52508e178b7, group_id=caade8d95a1d41749bb1950641ad6d48, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:53.369177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159be622-e6f4-42a8-b819-2147e4a35002 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:53.401916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 208/411] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 4 09:01:53 2022] PUT /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7/groups/caade8d95a1d41749bb1950641ad6d48/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:53.401916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ebc8e-e69e-4c41-a0dc-fcf1e17ef915 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.401916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ebc8e-e69e-4c41-a0dc-fcf1e17ef915 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.401916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ebc8e-e69e-4c41-a0dc-fcf1e17ef915 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.401916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 209/412] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:53.403497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ebc8e-e69e-4c41-a0dc-fcf1e17ef915 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:53.475693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f067396-3b7a-4251-b2d2-c3eb08401f53 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:53.494965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:53.524162 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:53.535696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6562d8-24b4-40b0-8d74-aa93d4939b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:53.537681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6562d8-24b4-40b0-8d74-aa93d4939b3a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.538202 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6562d8-24b4-40b0-8d74-aa93d4939b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.538649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6562d8-24b4-40b0-8d74-aa93d4939b3a None admin] PATH_INFO: `/v3/projects/3f0a53c420a84618ab851430c484608a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.541105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6562d8-24b4-40b0-8d74-aa93d4939b3a None admin] RBAC: Authorizing `identity:get_project(project_id=3f0a53c420a84618ab851430c484608a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:53.543558 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6562d8-24b4-40b0-8d74-aa93d4939b3a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:53.546167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 204/413] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:53 2022] GET /identity/v3/projects/3f0a53c420a84618ab851430c484608a => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:53.547444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ebc8e-e69e-4c41-a0dc-fcf1e17ef915 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:53.548020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd85c5-4ae5-4566-9c3a-e5d1c80218e6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.548479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd85c5-4ae5-4566-9c3a-e5d1c80218e6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.548918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd85c5-4ae5-4566-9c3a-e5d1c80218e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.571169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-fecd85c5-4ae5-4566-9c3a-e5d1c80218e6 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:53.572437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fecd85c5-4ae5-4566-9c3a-e5d1c80218e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:53.588575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2894cc6-66cf-4148-97fc-af85db800b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:53.589553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2894cc6-66cf-4148-97fc-af85db800b07 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.590622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2894cc6-66cf-4148-97fc-af85db800b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.590622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2894cc6-66cf-4148-97fc-af85db800b07 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.591664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2894cc6-66cf-4148-97fc-af85db800b07 None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:53.593061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2894cc6-66cf-4148-97fc-af85db800b07 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:53.595170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 210/414] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:53 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:53.595935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2894cc6-66cf-4148-97fc-af85db800b07 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:53.647772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 205/415] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:53.648827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd85c5-4ae5-4566-9c3a-e5d1c80218e6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:53.726832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2894cc6-66cf-4148-97fc-af85db800b07 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:53.754001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd85c5-4ae5-4566-9c3a-e5d1c80218e6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:53.761530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ba3ea-a0fd-46cb-aef9-ed65ac9a5db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:53.762546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ba3ea-a0fd-46cb-aef9-ed65ac9a5db0 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.762902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ba3ea-a0fd-46cb-aef9-ed65ac9a5db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.763256 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ba3ea-a0fd-46cb-aef9-ed65ac9a5db0 None admin] PATH_INFO: `/v3/projects/3f0a53c420a84618ab851430c484608a/groups/26a9090c30004262a57805eed2b95902/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.765282 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ba3ea-a0fd-46cb-aef9-ed65ac9a5db0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f0a53c420a84618ab851430c484608a, group_id=26a9090c30004262a57805eed2b95902, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:53.765674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:53.766899 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ba3ea-a0fd-46cb-aef9-ed65ac9a5db0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:53.789913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 211/416] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 4 09:01:53 2022] PUT /identity/v3/projects/3f0a53c420a84618ab851430c484608a/groups/26a9090c30004262a57805eed2b95902/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:53.789913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ba3ea-a0fd-46cb-aef9-ed65ac9a5db0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:53.817381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07898d-6e52-40b1-9e63-ec80a181e562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:53.818371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07898d-6e52-40b1-9e63-ec80a181e562 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.819534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07898d-6e52-40b1-9e63-ec80a181e562 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.819819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07898d-6e52-40b1-9e63-ec80a181e562 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7/groups/caade8d95a1d41749bb1950641ad6d48/roles/2eea147ab8b1426294a08106410f6d45` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.826270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:53.826665 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07898d-6e52-40b1-9e63-ec80a181e562 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e689a8addd941f8a2efd52508e178b7, group_id=caade8d95a1d41749bb1950641ad6d48, role_id=2eea147ab8b1426294a08106410f6d45)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:53.829769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07898d-6e52-40b1-9e63-ec80a181e562 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:53.853422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 206/417] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 4 09:01:53 2022] PUT /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7/groups/caade8d95a1d41749bb1950641ad6d48/roles/2eea147ab8b1426294a08106410f6d45 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:01:53.854670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699062d-c901-4b6a-b71b-755266499970 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.854963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699062d-c901-4b6a-b71b-755266499970 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.855614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699062d-c901-4b6a-b71b-755266499970 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.856639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 207/418] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:53.862415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f774f-ada0-40bc-bfea-17113c3aac45 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:53.862698 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f774f-ada0-40bc-bfea-17113c3aac45 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:53.862976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f774f-ada0-40bc-bfea-17113c3aac45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:53.881816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-231f774f-ada0-40bc-bfea-17113c3aac45 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:53.883111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-231f774f-ada0-40bc-bfea-17113c3aac45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:53.959995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ba3ea-a0fd-46cb-aef9-ed65ac9a5db0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:53.966052 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 208/419] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:53.990718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f774f-ada0-40bc-bfea-17113c3aac45 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:54.000750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3745df0-ffcd-4b55-a737-ce192bd0d7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:54.002801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3745df0-ffcd-4b55-a737-ce192bd0d7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:54.003192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3745df0-ffcd-4b55-a737-ce192bd0d7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:54.003570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3745df0-ffcd-4b55-a737-ce192bd0d7b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:54.019956 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3745df0-ffcd-4b55-a737-ce192bd0d7b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b3745df0-ffcd-4b55-a737-ce192bd0d7b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:54.030611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:54.031757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:54.032728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:54.035938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 212/420] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:54.046036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3745df0-ffcd-4b55-a737-ce192bd0d7b5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:54.131486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:54.136406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f774f-ada0-40bc-bfea-17113c3aac45 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:54.144199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:54.181687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fddaa-7cb4-4742-9cd3-db28a9a82bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:54.182809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fddaa-7cb4-4742-9cd3-db28a9a82bef None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:54.183099 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fddaa-7cb4-4742-9cd3-db28a9a82bef None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:54.183574 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fddaa-7cb4-4742-9cd3-db28a9a82bef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:54.188725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3745df0-ffcd-4b55-a737-ce192bd0d7b5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:54.199905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fddaa-7cb4-4742-9cd3-db28a9a82bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-255fddaa-7cb4-4742-9cd3-db28a9a82bef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:54.210469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:54.211396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:54.212212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:54.215060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 209/421] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:54.231783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fddaa-7cb4-4742-9cd3-db28a9a82bef None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:54.239222 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80161564-0c00-4690-8a40-949f1037c287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:54.239908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80161564-0c00-4690-8a40-949f1037c287 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:54.240239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80161564-0c00-4690-8a40-949f1037c287 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:54.240522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80161564-0c00-4690-8a40-949f1037c287 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:54.241345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80161564-0c00-4690-8a40-949f1037c287 None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:54.241600 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80161564-0c00-4690-8a40-949f1037c287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:54.248503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80161564-0c00-4690-8a40-949f1037c287 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:54.253946 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 213/422] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:54.259435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80161564-0c00-4690-8a40-949f1037c287 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:54.364862 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80161564-0c00-4690-8a40-949f1037c287 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:54.408663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863c8ef-afa5-4d8e-8a11-35dcbd8b1597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:54.409328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c8ef-afa5-4d8e-8a11-35dcbd8b1597 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:54.409613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c8ef-afa5-4d8e-8a11-35dcbd8b1597 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:54.409909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863c8ef-afa5-4d8e-8a11-35dcbd8b1597 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:54.417178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:54.478299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fddaa-7cb4-4742-9cd3-db28a9a82bef None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:54.535444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863c8ef-afa5-4d8e-8a11-35dcbd8b1597 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:54.536855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863c8ef-afa5-4d8e-8a11-35dcbd8b1597 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:54.542585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e4529-93ac-44aa-af5d-5c8269b7b91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:54.543269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e4529-93ac-44aa-af5d-5c8269b7b91c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:54.543567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e4529-93ac-44aa-af5d-5c8269b7b91c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:54.543844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e4529-93ac-44aa-af5d-5c8269b7b91c None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:54.544600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e4529-93ac-44aa-af5d-5c8269b7b91c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:54.544906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e4529-93ac-44aa-af5d-5c8269b7b91c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:54.546320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e4529-93ac-44aa-af5d-5c8269b7b91c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:54.558144 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 210/423] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:54.563723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e4529-93ac-44aa-af5d-5c8269b7b91c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:54.606457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:54.706487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e4529-93ac-44aa-af5d-5c8269b7b91c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:54.745651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083436c-53e4-4494-b2dc-e14fd8fa812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:54.746369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083436c-53e4-4494-b2dc-e14fd8fa812d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:54.746636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083436c-53e4-4494-b2dc-e14fd8fa812d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:54.746911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083436c-53e4-4494-b2dc-e14fd8fa812d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:54.759205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 214/424] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:54.766313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863c8ef-afa5-4d8e-8a11-35dcbd8b1597 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:54.832286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083436c-53e4-4494-b2dc-e14fd8fa812d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:54.833822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083436c-53e4-4494-b2dc-e14fd8fa812d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:54.837907 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:54.857258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:54.966538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 211/425] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:54.970472 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083436c-53e4-4494-b2dc-e14fd8fa812d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:54.987827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863c8ef-afa5-4d8e-8a11-35dcbd8b1597 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:55.051070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d613db-346c-40e7-ba28-fbd4c287d18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:55.052068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d613db-346c-40e7-ba28-fbd4c287d18a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:55.052353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d613db-346c-40e7-ba28-fbd4c287d18a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:55.052625 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d613db-346c-40e7-ba28-fbd4c287d18a None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:55.058694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d613db-346c-40e7-ba28-fbd4c287d18a None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:55.062287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d613db-346c-40e7-ba28-fbd4c287d18a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:55.062439 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 215/426] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:54 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:55.070399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d613db-346c-40e7-ba28-fbd4c287d18a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:55.152229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083436c-53e4-4494-b2dc-e14fd8fa812d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:55.177002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d613db-346c-40e7-ba28-fbd4c287d18a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:55.178621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:55.187383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:55.209143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eccc8a-16b2-4775-ac09-fd73728d6943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:55.214123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eccc8a-16b2-4775-ac09-fd73728d6943 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:55.214425 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eccc8a-16b2-4775-ac09-fd73728d6943 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:55.214711 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eccc8a-16b2-4775-ac09-fd73728d6943 None admin] PATH_INFO: `/v3/projects/795c9a5eff6345f6ae2de8e24f361e51` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:55.216025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eccc8a-16b2-4775-ac09-fd73728d6943 None admin] RBAC: Authorizing `identity:get_project(project_id=795c9a5eff6345f6ae2de8e24f361e51)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:55.217783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eccc8a-16b2-4775-ac09-fd73728d6943 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:55.219880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743feb10-3dc7-4e49-bb6b-d13edf9f14ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:55.220032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 212/427] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:54 2022] GET /identity/v3/projects/795c9a5eff6345f6ae2de8e24f361e51 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:55.221033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743feb10-3dc7-4e49-bb6b-d13edf9f14ea None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:55.221497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743feb10-3dc7-4e49-bb6b-d13edf9f14ea None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:55.221787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743feb10-3dc7-4e49-bb6b-d13edf9f14ea None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:55.225332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743feb10-3dc7-4e49-bb6b-d13edf9f14ea None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:55.226838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743feb10-3dc7-4e49-bb6b-d13edf9f14ea None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:55.228766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 216/428] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:55 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:55.229447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eccc8a-16b2-4775-ac09-fd73728d6943 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:55.245500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743feb10-3dc7-4e49-bb6b-d13edf9f14ea None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:55.341495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eccc8a-16b2-4775-ac09-fd73728d6943 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:55.382318 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accdc6d-9caf-427a-8070-e7e58f149aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:55.383314 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accdc6d-9caf-427a-8070-e7e58f149aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:55.385907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accdc6d-9caf-427a-8070-e7e58f149aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:55.385907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accdc6d-9caf-427a-8070-e7e58f149aa8 None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:55.385907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accdc6d-9caf-427a-8070-e7e58f149aa8 None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:55.386807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accdc6d-9caf-427a-8070-e7e58f149aa8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:55.388733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 213/429] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:55 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:55.395109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accdc6d-9caf-427a-8070-e7e58f149aa8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:55.488994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743feb10-3dc7-4e49-bb6b-d13edf9f14ea None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:55.499981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:55.553755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fdd63-fc40-4e00-86d0-a8643c056bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:55.554568 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fdd63-fc40-4e00-86d0-a8643c056bfa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:55.554861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fdd63-fc40-4e00-86d0-a8643c056bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:55.555140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fdd63-fc40-4e00-86d0-a8643c056bfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:55.555987 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fdd63-fc40-4e00-86d0-a8643c056bfa None admin] RBAC: Adding query filter params (group.id=caade8d95a1d41749bb1950641ad6d48, role.id=0b98291295724747acd01dd0d2306e62, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:55.556301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fdd63-fc40-4e00-86d0-a8643c056bfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:55.557764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fdd63-fc40-4e00-86d0-a8643c056bfa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:55.570061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 217/430] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 4 09:01:55 2022] GET /identity/v3/role_assignments?group.id=caade8d95a1d41749bb1950641ad6d48&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=0b98291295724747acd01dd0d2306e62 => generated 611 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:55.584825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accdc6d-9caf-427a-8070-e7e58f149aa8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:55.633586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc167dc9-18fa-41cc-9f52-191bb3a74061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:55.634291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc167dc9-18fa-41cc-9f52-191bb3a74061 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:55.634577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc167dc9-18fa-41cc-9f52-191bb3a74061 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:55.634858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc167dc9-18fa-41cc-9f52-191bb3a74061 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:55.635728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc167dc9-18fa-41cc-9f52-191bb3a74061 None admin] RBAC: Adding query filter params (group.id=caade8d95a1d41749bb1950641ad6d48, role.id=2eea147ab8b1426294a08106410f6d45, scope.project.id=795c9a5eff6345f6ae2de8e24f361e51) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:55.636041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc167dc9-18fa-41cc-9f52-191bb3a74061 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:55.637490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc167dc9-18fa-41cc-9f52-191bb3a74061 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:55.640764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65784f05-2756-4765-827a-32da3c928e28 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:55.641061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65784f05-2756-4765-827a-32da3c928e28 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:55.641343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65784f05-2756-4765-827a-32da3c928e28 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:55.643121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 218/431] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:01:55.647380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 214/432] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 4 09:01:55 2022] GET /identity/v3/role_assignments?group.id=caade8d95a1d41749bb1950641ad6d48&scope.project.id=795c9a5eff6345f6ae2de8e24f361e51&role.id=2eea147ab8b1426294a08106410f6d45 => generated 611 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:55.648860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db500c-a345-47e1-8449-ef28955d04a3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:55.649155 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db500c-a345-47e1-8449-ef28955d04a3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:55.649438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db500c-a345-47e1-8449-ef28955d04a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:55.669178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b5db500c-a345-47e1-8449-ef28955d04a3 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:55.670494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b5db500c-a345-47e1-8449-ef28955d04a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:55.722809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:55.792377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 219/433] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:55.838239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc167dc9-18fa-41cc-9f52-191bb3a74061 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:55.956906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc167dc9-18fa-41cc-9f52-191bb3a74061 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:55.979418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:55.994439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5067e0a-e3cb-416a-83c5-caea7d9b87a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:55.995252 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5067e0a-e3cb-416a-83c5-caea7d9b87a7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:55.995533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5067e0a-e3cb-416a-83c5-caea7d9b87a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:55.995807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5067e0a-e3cb-416a-83c5-caea7d9b87a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.007323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5067e0a-e3cb-416a-83c5-caea7d9b87a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-a5067e0a-e3cb-416a-83c5-caea7d9b87a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:56.014973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:56.015902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:56.016729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 215/434] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:56.022315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db500c-a345-47e1-8449-ef28955d04a3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:56.128678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db500c-a345-47e1-8449-ef28955d04a3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:56.168366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556b7a9-597a-4421-a402-a67f5b194824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:56.169360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b7a9-597a-4421-a402-a67f5b194824 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.169772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b7a9-597a-4421-a402-a67f5b194824 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.170299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b7a9-597a-4421-a402-a67f5b194824 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.171204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556b7a9-597a-4421-a402-a67f5b194824 None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:56.171740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556b7a9-597a-4421-a402-a67f5b194824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:56.173266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556b7a9-597a-4421-a402-a67f5b194824 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:56.183101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 220/435] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:01:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:56.188600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5067e0a-e3cb-416a-83c5-caea7d9b87a7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:56.275377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef7869-21fc-43bf-a31d-816325785f96 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.276070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef7869-21fc-43bf-a31d-816325785f96 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.276597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef7869-21fc-43bf-a31d-816325785f96 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.278513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 221/436] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:56.285067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcced33-1da9-49b9-a441-e2f61fb43a67 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.285362 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcced33-1da9-49b9-a441-e2f61fb43a67 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.285641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcced33-1da9-49b9-a441-e2f61fb43a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.303423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bbcced33-1da9-49b9-a441-e2f61fb43a67 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:56.304933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bbcced33-1da9-49b9-a441-e2f61fb43a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:56.306256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5067e0a-e3cb-416a-83c5-caea7d9b87a7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:56.332476 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:56.346509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f274f5-4b48-4011-a694-4dd3fdaf989e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:56.347168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f274f5-4b48-4011-a694-4dd3fdaf989e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.347485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f274f5-4b48-4011-a694-4dd3fdaf989e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.347772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f274f5-4b48-4011-a694-4dd3fdaf989e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.388269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 222/437] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:01:56.395534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dfb10-58ec-4eb9-ae15-4a0769e33229 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.395822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dfb10-58ec-4eb9-ae15-4a0769e33229 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.396099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dfb10-58ec-4eb9-ae15-4a0769e33229 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.397442 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 223/438] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:01:56.403053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7367e6-505a-4d44-baac-b24b4565eb21 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.403347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7367e6-505a-4d44-baac-b24b4565eb21 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.403628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7367e6-505a-4d44-baac-b24b4565eb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.408362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:56.422637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9b7367e6-505a-4d44-baac-b24b4565eb21 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:01:56.423866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9b7367e6-505a-4d44-baac-b24b4565eb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:01:56.430703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f274f5-4b48-4011-a694-4dd3fdaf989e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:56.432600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f274f5-4b48-4011-a694-4dd3fdaf989e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:56.494175 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 224/439] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:01:56.495113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7367e6-505a-4d44-baac-b24b4565eb21 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:56.530367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 216/440] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:56.531209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f274f5-4b48-4011-a694-4dd3fdaf989e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:56.583626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:56.598697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7367e6-505a-4d44-baac-b24b4565eb21 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:56.610893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:56.631041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f274f5-4b48-4011-a694-4dd3fdaf989e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:56.633694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10009c04-9053-4384-a557-0ffa9bc2dc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:56.634530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10009c04-9053-4384-a557-0ffa9bc2dc28 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.634819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10009c04-9053-4384-a557-0ffa9bc2dc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.635100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10009c04-9053-4384-a557-0ffa9bc2dc28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.645783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10009c04-9053-4384-a557-0ffa9bc2dc28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-10009c04-9053-4384-a557-0ffa9bc2dc28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:56.653375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:56.654521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 225/441] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:01:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:56.655342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10009c04-9053-4384-a557-0ffa9bc2dc28 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:56.668585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58086305-6e08-436a-a9bc-3184b00bd833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:56.669401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58086305-6e08-436a-a9bc-3184b00bd833 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.669682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58086305-6e08-436a-a9bc-3184b00bd833 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.669985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58086305-6e08-436a-a9bc-3184b00bd833 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.681015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58086305-6e08-436a-a9bc-3184b00bd833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-58086305-6e08-436a-a9bc-3184b00bd833 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:01:56.688931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:01:56.689813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:01:56.690662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 217/442] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:01:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:01:56.691383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58086305-6e08-436a-a9bc-3184b00bd833 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:56.757215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10009c04-9053-4384-a557-0ffa9bc2dc28 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:56.789621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ce81-aa2e-432d-b514-213059bfb82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:56.790512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ce81-aa2e-432d-b514-213059bfb82d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.790686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58086305-6e08-436a-a9bc-3184b00bd833 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:56.790815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ce81-aa2e-432d-b514-213059bfb82d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.791019 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ce81-aa2e-432d-b514-213059bfb82d None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.792242 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08ce81-aa2e-432d-b514-213059bfb82d None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:56.793674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08ce81-aa2e-432d-b514-213059bfb82d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:56.795400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 226/443] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:56 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:56.796150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ce81-aa2e-432d-b514-213059bfb82d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:56.815947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:56.824202 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aae69-42a4-4832-a078-0a41874dd67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:56.824903 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aae69-42a4-4832-a078-0a41874dd67a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.825189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aae69-42a4-4832-a078-0a41874dd67a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.825467 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aae69-42a4-4832-a078-0a41874dd67a None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.826255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15aae69-42a4-4832-a078-0a41874dd67a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:56.826564 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15aae69-42a4-4832-a078-0a41874dd67a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:56.827956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15aae69-42a4-4832-a078-0a41874dd67a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:56.836452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 218/444] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:01:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:56.837237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aae69-42a4-4832-a078-0a41874dd67a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:56.839785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:56.901834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ce81-aa2e-432d-b514-213059bfb82d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:56.934882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3babb3f-e4a1-4905-ac79-0556f0718afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:56.935549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3babb3f-e4a1-4905-ac79-0556f0718afd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.935646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aae69-42a4-4832-a078-0a41874dd67a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:56.935834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3babb3f-e4a1-4905-ac79-0556f0718afd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.936110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3babb3f-e4a1-4905-ac79-0556f0718afd None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.936860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3babb3f-e4a1-4905-ac79-0556f0718afd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:56.937171 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3babb3f-e4a1-4905-ac79-0556f0718afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:56.938581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3babb3f-e4a1-4905-ac79-0556f0718afd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:56.946770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 227/445] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:01:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:56.947357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3babb3f-e4a1-4905-ac79-0556f0718afd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:56.973844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed0e1f-eca3-4be8-9efe-f6eb1ec539d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:56.974725 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed0e1f-eca3-4be8-9efe-f6eb1ec539d0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:56.975003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed0e1f-eca3-4be8-9efe-f6eb1ec539d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:56.975301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed0e1f-eca3-4be8-9efe-f6eb1ec539d0 None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:56.976544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed0e1f-eca3-4be8-9efe-f6eb1ec539d0 None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:56.978083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed0e1f-eca3-4be8-9efe-f6eb1ec539d0 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:56.984757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 219/446] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:56 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:56.984757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed0e1f-eca3-4be8-9efe-f6eb1ec539d0 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:57.035391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:57.052456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:57.059311 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3babb3f-e4a1-4905-ac79-0556f0718afd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:57.084682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed0e1f-eca3-4be8-9efe-f6eb1ec539d0 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:57.091695 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b58020-d6d7-4e16-91bb-3c4d179ec8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:57.092351 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b58020-d6d7-4e16-91bb-3c4d179ec8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:57.092638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b58020-d6d7-4e16-91bb-3c4d179ec8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:57.092917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b58020-d6d7-4e16-91bb-3c4d179ec8b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:57.117514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cb6ff-a7e1-4eff-96e6-afe8b4bb2e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:57.118293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cb6ff-a7e1-4eff-96e6-afe8b4bb2e75 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:57.118580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cb6ff-a7e1-4eff-96e6-afe8b4bb2e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:57.118857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cb6ff-a7e1-4eff-96e6-afe8b4bb2e75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:57.164011 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b58020-d6d7-4e16-91bb-3c4d179ec8b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:57.165498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b58020-d6d7-4e16-91bb-3c4d179ec8b3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:57.187415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cb6ff-a7e1-4eff-96e6-afe8b4bb2e75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:57.188881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cb6ff-a7e1-4eff-96e6-afe8b4bb2e75 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:57.241145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:57.257990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:57.264664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 228/447] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:57.265509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b58020-d6d7-4e16-91bb-3c4d179ec8b3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:57.295052 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 220/448] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:01:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:01:57.295888 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cb6ff-a7e1-4eff-96e6-afe8b4bb2e75 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:57.366104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b58020-d6d7-4e16-91bb-3c4d179ec8b3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:57.396605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cb6ff-a7e1-4eff-96e6-afe8b4bb2e75 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:57.400740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c601d-093d-4e4a-ac14-91f095fe8ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:57.401400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c601d-093d-4e4a-ac14-91f095fe8ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:57.401686 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c601d-093d-4e4a-ac14-91f095fe8ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:57.401995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c601d-093d-4e4a-ac14-91f095fe8ed3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:57.402835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c601d-093d-4e4a-ac14-91f095fe8ed3 None admin] RBAC: Adding query filter params (group.id=caade8d95a1d41749bb1950641ad6d48, role.id=0b98291295724747acd01dd0d2306e62, scope.project.id=7e689a8addd941f8a2efd52508e178b7) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:57.403149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c601d-093d-4e4a-ac14-91f095fe8ed3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:57.404665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c601d-093d-4e4a-ac14-91f095fe8ed3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:57.410951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 229/449] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 4 09:01:57 2022] GET /identity/v3/role_assignments?group.id=caade8d95a1d41749bb1950641ad6d48&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=0b98291295724747acd01dd0d2306e62 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:57.411782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c601d-093d-4e4a-ac14-91f095fe8ed3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:57.450680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58957e-5fc0-4184-add8-3b6852a3ffd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:57.451559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58957e-5fc0-4184-add8-3b6852a3ffd9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:57.451842 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58957e-5fc0-4184-add8-3b6852a3ffd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:57.452117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58957e-5fc0-4184-add8-3b6852a3ffd9 None admin] PATH_INFO: `/v3/projects/3f0a53c420a84618ab851430c484608a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:57.453335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58957e-5fc0-4184-add8-3b6852a3ffd9 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0a53c420a84618ab851430c484608a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:57.455386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58957e-5fc0-4184-add8-3b6852a3ffd9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:57.456226 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:57.457748 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 221/450] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:57 2022] GET /identity/v3/projects/3f0a53c420a84618ab851430c484608a => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:57.463704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58957e-5fc0-4184-add8-3b6852a3ffd9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:57.508302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:57.528061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c601d-093d-4e4a-ac14-91f095fe8ed3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:57.569410 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d21547-d1bb-47f8-890d-fbe783609cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:57.569691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d21547-d1bb-47f8-890d-fbe783609cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:57.570018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d21547-d1bb-47f8-890d-fbe783609cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:57.570297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d21547-d1bb-47f8-890d-fbe783609cc1 None admin] PATH_INFO: `/v3/projects/7e689a8addd941f8a2efd52508e178b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:57.571515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d21547-d1bb-47f8-890d-fbe783609cc1 None admin] RBAC: Authorizing `identity:get_project(project_id=7e689a8addd941f8a2efd52508e178b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:57.572973 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d21547-d1bb-47f8-890d-fbe783609cc1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:57.575207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 230/451] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:01:57 2022] GET /identity/v3/projects/7e689a8addd941f8a2efd52508e178b7 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:57.578183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58957e-5fc0-4184-add8-3b6852a3ffd9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:57.581223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d21547-d1bb-47f8-890d-fbe783609cc1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:57.618575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b44766-39a9-4ff2-bc7c-1d60b83f044a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:57.619437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b44766-39a9-4ff2-bc7c-1d60b83f044a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:57.621801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b44766-39a9-4ff2-bc7c-1d60b83f044a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:57.621801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b44766-39a9-4ff2-bc7c-1d60b83f044a None admin] PATH_INFO: `/v3/groups/26a9090c30004262a57805eed2b95902` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:57.621801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b44766-39a9-4ff2-bc7c-1d60b83f044a None admin] RBAC: Authorizing `identity:get_group(group_id=26a9090c30004262a57805eed2b95902)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:57.622628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b44766-39a9-4ff2-bc7c-1d60b83f044a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:57.629504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 222/452] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:57 2022] GET /identity/v3/groups/26a9090c30004262a57805eed2b95902 => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:57.630777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b44766-39a9-4ff2-bc7c-1d60b83f044a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:57.706912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d21547-d1bb-47f8-890d-fbe783609cc1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:57.717866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:57.740795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b44766-39a9-4ff2-bc7c-1d60b83f044a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:57.747030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4a0d0e-86c4-4bbf-964f-4706a3a9e5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:57.747890 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a0d0e-86c4-4bbf-964f-4706a3a9e5bd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:57.748172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a0d0e-86c4-4bbf-964f-4706a3a9e5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:57.748440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a0d0e-86c4-4bbf-964f-4706a3a9e5bd None admin] PATH_INFO: `/v3/groups/caade8d95a1d41749bb1950641ad6d48` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:57.749152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:01:57.750626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4a0d0e-86c4-4bbf-964f-4706a3a9e5bd None admin] RBAC: Authorizing `identity:get_group(group_id=caade8d95a1d41749bb1950641ad6d48)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:57.752410 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4a0d0e-86c4-4bbf-964f-4706a3a9e5bd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:57.754356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 231/453] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 4 09:01:57 2022] GET /identity/v3/groups/caade8d95a1d41749bb1950641ad6d48 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:01:57.760305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4a0d0e-86c4-4bbf-964f-4706a3a9e5bd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:01:57.780998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c889449-e38c-473c-b19f-c3ad4ebbefc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:57.781648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c889449-e38c-473c-b19f-c3ad4ebbefc1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:57.782022 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c889449-e38c-473c-b19f-c3ad4ebbefc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:57.782301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c889449-e38c-473c-b19f-c3ad4ebbefc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:57.783134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c889449-e38c-473c-b19f-c3ad4ebbefc1 None admin] RBAC: Adding query filter params (group.id=26a9090c30004262a57805eed2b95902, role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=3f0a53c420a84618ab851430c484608a) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:57.783446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c889449-e38c-473c-b19f-c3ad4ebbefc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:57.784869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c889449-e38c-473c-b19f-c3ad4ebbefc1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:57.790857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 223/454] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 4 09:01:57 2022] GET /identity/v3/role_assignments?group.id=26a9090c30004262a57805eed2b95902&scope.project.id=3f0a53c420a84618ab851430c484608a&role.id=2302517080ac42578aa81c606dd0ff2f => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:01:57.866857 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4a0d0e-86c4-4bbf-964f-4706a3a9e5bd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:01:57.904303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae1c31-5402-4dbf-9564-38ff1df065bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:01:57.904987 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae1c31-5402-4dbf-9564-38ff1df065bf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:01:57.905275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae1c31-5402-4dbf-9564-38ff1df065bf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:01:57.905539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae1c31-5402-4dbf-9564-38ff1df065bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:01:57.906427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae1c31-5402-4dbf-9564-38ff1df065bf None admin] RBAC: Adding query filter params (group.id=caade8d95a1d41749bb1950641ad6d48, role.id=2eea147ab8b1426294a08106410f6d45, scope.project.id=7e689a8addd941f8a2efd52508e178b7) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:01:57.906741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae1c31-5402-4dbf-9564-38ff1df065bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:01:57.908213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae1c31-5402-4dbf-9564-38ff1df065bf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:01:57.914903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 232/455] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 4 09:01:57 2022] GET /identity/v3/role_assignments?group.id=caade8d95a1d41749bb1950641ad6d48&scope.project.id=7e689a8addd941f8a2efd52508e178b7&role.id=2eea147ab8b1426294a08106410f6d45 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:00.468752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d656de-3530-4738-9f01-b82b35602e3b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:00.469271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d656de-3530-4738-9f01-b82b35602e3b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:00.469271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d656de-3530-4738-9f01-b82b35602e3b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:00.470870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 224/456] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:00.476798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d22436-9e0f-4945-91b6-d389c7bce60d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:00.477235 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d22436-9e0f-4945-91b6-d389c7bce60d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:00.477405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d22436-9e0f-4945-91b6-d389c7bce60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:00.502377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-87d22436-9e0f-4945-91b6-d389c7bce60d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:00.503577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-87d22436-9e0f-4945-91b6-d389c7bce60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:00.555474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:00.599477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 233/457] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:02:00.636632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d656de-3530-4738-9f01-b82b35602e3b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:00.752804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db1825-5a1b-4fe3-a229-f6d4486335bd None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:00.753102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db1825-5a1b-4fe3-a229-f6d4486335bd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:00.753283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db1825-5a1b-4fe3-a229-f6d4486335bd None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:00.755153 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 234/458] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:00.760363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fdb81-b712-4d5d-8aaa-579767175b50 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:00.760657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fdb81-b712-4d5d-8aaa-579767175b50 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:00.760942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fdb81-b712-4d5d-8aaa-579767175b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:00.790262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-998fdb81-b712-4d5d-8aaa-579767175b50 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:00.791473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-998fdb81-b712-4d5d-8aaa-579767175b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:00.802927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d656de-3530-4738-9f01-b82b35602e3b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:00.838815 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2727915-8ac8-4867-b79d-cd0e43482dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:00.840001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2727915-8ac8-4867-b79d-cd0e43482dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:00.840479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2727915-8ac8-4867-b79d-cd0e43482dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:00.840982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2727915-8ac8-4867-b79d-cd0e43482dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:00.852792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2727915-8ac8-4867-b79d-cd0e43482dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:00.854699 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2727915-8ac8-4867-b79d-cd0e43482dd6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:00.868567 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 235/459] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:02:00.869734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f35ca7-1f41-4549-8323-867c7035645b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-b2727915-8ac8-4867-b79d-cd0e43482dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:00.874000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:00.874881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:00.875772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:00.875772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:00.875772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:00.875772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:00.878177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f35ca7-1f41-4549-8323-867c7035645b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:00.878390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f35ca7-1f41-4549-8323-867c7035645b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:00.878485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 225/460] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:00.879864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 236/461] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:00 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:00.884771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2727915-8ac8-4867-b79d-cd0e43482dd6 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:00.885151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c639dd-dd56-4665-90a8-e834bc5f4d1a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:00.885445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c639dd-dd56-4665-90a8-e834bc5f4d1a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:00.885730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c639dd-dd56-4665-90a8-e834bc5f4d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:00.915673 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-52c639dd-dd56-4665-90a8-e834bc5f4d1a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:00.916895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-52c639dd-dd56-4665-90a8-e834bc5f4d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:00.917751 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:00.987022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 237/462] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:00.993902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c639dd-dd56-4665-90a8-e834bc5f4d1a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:01.011821 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2727915-8ac8-4867-b79d-cd0e43482dd6 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:01.018731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:01.057464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee47dd-70b7-41ce-90e2-1c889b74f98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:01.058535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee47dd-70b7-41ce-90e2-1c889b74f98a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:01.059014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee47dd-70b7-41ce-90e2-1c889b74f98a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:01.059506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee47dd-70b7-41ce-90e2-1c889b74f98a None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:01.060468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee47dd-70b7-41ce-90e2-1c889b74f98a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:01.060971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee47dd-70b7-41ce-90e2-1c889b74f98a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:01.062613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee47dd-70b7-41ce-90e2-1c889b74f98a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:01.078687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 226/463] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:01.079897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee47dd-70b7-41ce-90e2-1c889b74f98a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:01.095853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c639dd-dd56-4665-90a8-e834bc5f4d1a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:01.136588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e70c4d-05ec-4f91-a3d7-c2b8714c3f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:01.137403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e70c4d-05ec-4f91-a3d7-c2b8714c3f2c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:01.137692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e70c4d-05ec-4f91-a3d7-c2b8714c3f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:01.138059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e70c4d-05ec-4f91-a3d7-c2b8714c3f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:01.148824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e70c4d-05ec-4f91-a3d7-c2b8714c3f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:01.152213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e70c4d-05ec-4f91-a3d7-c2b8714c3f2c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f9e70c4d-05ec-4f91-a3d7-c2b8714c3f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:01.169607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:01.170583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:01.171506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:01.171506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:01.171506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:01.171506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:01.171506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 238/464] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:01.171781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e70c4d-05ec-4f91-a3d7-c2b8714c3f2c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:01.179026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:01.199346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee47dd-70b7-41ce-90e2-1c889b74f98a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:01.236453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a718d6-7f8a-4d3a-a01f-36771e58b0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:01.237246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a718d6-7f8a-4d3a-a01f-36771e58b0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:01.237522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a718d6-7f8a-4d3a-a01f-36771e58b0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:01.237793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a718d6-7f8a-4d3a-a01f-36771e58b0b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:01.248765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a718d6-7f8a-4d3a-a01f-36771e58b0b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:01.250293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a718d6-7f8a-4d3a-a01f-36771e58b0b8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:01.262962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-a0a718d6-7f8a-4d3a-a01f-36771e58b0b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:01.267977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:01.268861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:01.270084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:01.270084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:01.270084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:01.270084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:01.270084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 227/465] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:01.271404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d40f1e-e1dc-45dc-ac3c-04fcc7f74464 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:01.271517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d40f1e-e1dc-45dc-ac3c-04fcc7f74464 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:01.271719 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d40f1e-e1dc-45dc-ac3c-04fcc7f74464 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:01.273216 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 228/466] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:01.273714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d40f1e-e1dc-45dc-ac3c-04fcc7f74464 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:01.291645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e70c4d-05ec-4f91-a3d7-c2b8714c3f2c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:01.330319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e57c8-3a49-496b-8a35-5f90287f462b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:01.331110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e57c8-3a49-496b-8a35-5f90287f462b None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:01.331396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e57c8-3a49-496b-8a35-5f90287f462b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:01.331676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e57c8-3a49-496b-8a35-5f90287f462b None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:01.332534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e57c8-3a49-496b-8a35-5f90287f462b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:01.334006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e57c8-3a49-496b-8a35-5f90287f462b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:01.366281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 239/467] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:02:01 2022] POST /identity/v3/users => generated 246 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:01.367138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e57c8-3a49-496b-8a35-5f90287f462b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:01.388642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d40f1e-e1dc-45dc-ac3c-04fcc7f74464 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:01.429855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645c753-1c49-4035-9095-79947c006d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:01.430838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645c753-1c49-4035-9095-79947c006d00 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:01.430838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645c753-1c49-4035-9095-79947c006d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:01.431113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645c753-1c49-4035-9095-79947c006d00 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:01.431878 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645c753-1c49-4035-9095-79947c006d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:01.432178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645c753-1c49-4035-9095-79947c006d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:01.433715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645c753-1c49-4035-9095-79947c006d00 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:01.436102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:01.448817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 229/468] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:01.450349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9c3eb-3991-4e89-88f9-ceb7279e9424 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:01.450646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9c3eb-3991-4e89-88f9-ceb7279e9424 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:01.450929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9c3eb-3991-4e89-88f9-ceb7279e9424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:01.469453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-20a9c3eb-3991-4e89-88f9-ceb7279e9424 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:01.470730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-20a9c3eb-3991-4e89-88f9-ceb7279e9424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:01.518542 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e57c8-3a49-496b-8a35-5f90287f462b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:01.557555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 230/469] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:01.558449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9c3eb-3991-4e89-88f9-ceb7279e9424 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:01.569195 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa298051-72b1-463d-ad44-6d0c419c3005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:01.570322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa298051-72b1-463d-ad44-6d0c419c3005 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:01.570804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa298051-72b1-463d-ad44-6d0c419c3005 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:01.571272 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa298051-72b1-463d-ad44-6d0c419c3005 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:01.572313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa298051-72b1-463d-ad44-6d0c419c3005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:01.572922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa298051-72b1-463d-ad44-6d0c419c3005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:01.574676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa298051-72b1-463d-ad44-6d0c419c3005 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:01.585116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:01.594220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 240/470] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:01.594400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa298051-72b1-463d-ad44-6d0c419c3005 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:01.676540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9c3eb-3991-4e89-88f9-ceb7279e9424 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:01.710496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f6fa5-e33a-46b8-bd39-afaf09f77cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:01.711205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f6fa5-e33a-46b8-bd39-afaf09f77cc3 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:01.711494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f6fa5-e33a-46b8-bd39-afaf09f77cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:01.711780 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f6fa5-e33a-46b8-bd39-afaf09f77cc3 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:01.712495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f6fa5-e33a-46b8-bd39-afaf09f77cc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:01.712953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:01.714686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f6fa5-e33a-46b8-bd39-afaf09f77cc3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:01.721344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa298051-72b1-463d-ad44-6d0c419c3005 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:01.744003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 231/471] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:02:01 2022] POST /identity/v3/users => generated 248 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:01.745254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f6fa5-e33a-46b8-bd39-afaf09f77cc3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:01.764689 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80944b7f-0d27-4bbb-bde0-7f1182f00033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:01.765497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80944b7f-0d27-4bbb-bde0-7f1182f00033 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:01.765843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80944b7f-0d27-4bbb-bde0-7f1182f00033 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:01.766253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80944b7f-0d27-4bbb-bde0-7f1182f00033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:01.785495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80944b7f-0d27-4bbb-bde0-7f1182f00033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:01.786992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80944b7f-0d27-4bbb-bde0-7f1182f00033 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-80944b7f-0d27-4bbb-bde0-7f1182f00033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:01.807730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:01.808655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:01.810105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:01.810105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:01.810105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:01.810105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:01.810105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 241/472] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:01.816719 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80944b7f-0d27-4bbb-bde0-7f1182f00033 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:01.877294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f6fa5-e33a-46b8-bd39-afaf09f77cc3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:01.919446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49854be0-6cea-4e09-b3bd-b5e5fabdb4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:01.920135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49854be0-6cea-4e09-b3bd-b5e5fabdb4c4 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:01.920422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49854be0-6cea-4e09-b3bd-b5e5fabdb4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:01.921197 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49854be0-6cea-4e09-b3bd-b5e5fabdb4c4 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:01.921971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49854be0-6cea-4e09-b3bd-b5e5fabdb4c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:01.923627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49854be0-6cea-4e09-b3bd-b5e5fabdb4c4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:01.930662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:01.950714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 232/473] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:02:01 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:02.001258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80944b7f-0d27-4bbb-bde0-7f1182f00033 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:02.041196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc752f2-4cdf-4349-8b5d-9fbd358a6c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:02.042184 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc752f2-4cdf-4349-8b5d-9fbd358a6c91 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:02.042658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc752f2-4cdf-4349-8b5d-9fbd358a6c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:02.043078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc752f2-4cdf-4349-8b5d-9fbd358a6c91 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:02.043981 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc752f2-4cdf-4349-8b5d-9fbd358a6c91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:02.044461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc752f2-4cdf-4349-8b5d-9fbd358a6c91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:02.046058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc752f2-4cdf-4349-8b5d-9fbd358a6c91 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:02.060134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 242/474] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:02.065439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49854be0-6cea-4e09-b3bd-b5e5fabdb4c4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:02.110357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:02.190145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49854be0-6cea-4e09-b3bd-b5e5fabdb4c4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:02.226194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5431dc-24a6-4e04-b51a-627ef3487468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:02.226876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5431dc-24a6-4e04-b51a-627ef3487468 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:02.227158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5431dc-24a6-4e04-b51a-627ef3487468 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:02.227439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5431dc-24a6-4e04-b51a-627ef3487468 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:02.228145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5431dc-24a6-4e04-b51a-627ef3487468 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:02.229598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5431dc-24a6-4e04-b51a-627ef3487468 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:02.252556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 233/475] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:02:02 2022] POST /identity/v3/users => generated 249 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:02:02.316322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3fd76-f56f-49a9-acf0-b596382744ad None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:02.316547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3fd76-f56f-49a9-acf0-b596382744ad None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:02.316836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3fd76-f56f-49a9-acf0-b596382744ad None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:02.318253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 243/476] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:02.324117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cc0e6-55c9-455f-a14a-26d2ad0c7f24 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:02.324432 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cc0e6-55c9-455f-a14a-26d2ad0c7f24 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:02.324685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cc0e6-55c9-455f-a14a-26d2ad0c7f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:02.344004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2c3cc0e6-55c9-455f-a14a-26d2ad0c7f24 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:02.345224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2c3cc0e6-55c9-455f-a14a-26d2ad0c7f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:02.442710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 234/477] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:02.468938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3fd76-f56f-49a9-acf0-b596382744ad None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:02.584901 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3fd76-f56f-49a9-acf0-b596382744ad None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:02.596352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:02.623246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e2286-96a9-4c99-b49a-2dd459f73661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:02.623932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2286-96a9-4c99-b49a-2dd459f73661 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:02.624219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2286-96a9-4c99-b49a-2dd459f73661 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:02.624497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e2286-96a9-4c99-b49a-2dd459f73661 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:02.625270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e2286-96a9-4c99-b49a-2dd459f73661 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:02.626680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e2286-96a9-4c99-b49a-2dd459f73661 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:02.636134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.sql.core [None req-eb7e2286-96a9-4c99-b49a-2dd459f73661 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 09:02:02.636134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 09:02:02.636134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [parameters: {'id': 'c84f71e0c2a54e3eb7f8876b1054b9c4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 09:02:02.636134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80284) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-eb7e2286-96a9-4c99-b49a-2dd459f73661 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jul 04 09:02:02.654846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return ref.to_dict() Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:02:02.655755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application yield resource Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise self.value Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application session.commit() Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 09:02:02.657119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.flush() Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application compat.raise_( Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 09:02:02.658777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:02:02.660896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:02:02.666480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [parameters: {'id': 'c84f71e0c2a54e3eb7f8876b1054b9c4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:02.667523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 04 09:02:02.668565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:02.669559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 244/478] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 4 09:02:02 2022] POST /identity/v3/roles => generated 144 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:02.669627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cc0e6-55c9-455f-a14a-26d2ad0c7f24 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:02.706255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:02.800668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cc0e6-55c9-455f-a14a-26d2ad0c7f24 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:02.845345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c585b74-ac33-45e9-a665-d9df6a421dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:02.846111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c585b74-ac33-45e9-a665-d9df6a421dbc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:02.846532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c585b74-ac33-45e9-a665-d9df6a421dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:02.846841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c585b74-ac33-45e9-a665-d9df6a421dbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:02.861828 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c585b74-ac33-45e9-a665-d9df6a421dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-9c585b74-ac33-45e9-a665-d9df6a421dbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:02.869976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:02.871071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:02.872109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 235/479] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:02:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:02.880135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e2286-96a9-4c99-b49a-2dd459f73661 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:03.006820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e2286-96a9-4c99-b49a-2dd459f73661 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:03.053693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee14e4-6779-4bf2-a508-d1cb59dd06e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:03.054405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee14e4-6779-4bf2-a508-d1cb59dd06e3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:03.054689 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee14e4-6779-4bf2-a508-d1cb59dd06e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:03.054968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee14e4-6779-4bf2-a508-d1cb59dd06e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:03.055735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee14e4-6779-4bf2-a508-d1cb59dd06e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:03.056045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee14e4-6779-4bf2-a508-d1cb59dd06e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:03.057434 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee14e4-6779-4bf2-a508-d1cb59dd06e3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:03.069101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 245/480] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:03.933840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd67d-162d-4e8e-a0bc-5b70550f9b7e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:03.934452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd67d-162d-4e8e-a0bc-5b70550f9b7e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:03.934452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd67d-162d-4e8e-a0bc-5b70550f9b7e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:03.936044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 236/481] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:03.941613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7b4a3-6946-4661-9226-c5b03c7ebc0e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:03.942701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7b4a3-6946-4661-9226-c5b03c7ebc0e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:03.942701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7b4a3-6946-4661-9226-c5b03c7ebc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:03.966980 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-95a7b4a3-6946-4661-9226-c5b03c7ebc0e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:03.968227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-95a7b4a3-6946-4661-9226-c5b03c7ebc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:04.007472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:04.067618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 246/482] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:02:04.090994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cd67d-162d-4e8e-a0bc-5b70550f9b7e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:04.229032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:04.254699 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cd67d-162d-4e8e-a0bc-5b70550f9b7e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:04.302610 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2fbbe-fbd9-4776-974f-012a55cd9728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:04.303422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2fbbe-fbd9-4776-974f-012a55cd9728 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:04.303703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2fbbe-fbd9-4776-974f-012a55cd9728 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:04.303978 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2fbbe-fbd9-4776-974f-012a55cd9728 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:04.319104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2fbbe-fbd9-4776-974f-012a55cd9728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-a0f2fbbe-fbd9-4776-974f-012a55cd9728 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:04.326951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:04.327961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:04.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:04.333006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 237/483] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:04.336601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7b4a3-6946-4661-9226-c5b03c7ebc0e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:04.366208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca9c1-2cd5-47ce-82ec-e5ff00ac36f7 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:04.366498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca9c1-2cd5-47ce-82ec-e5ff00ac36f7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:04.366780 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca9c1-2cd5-47ce-82ec-e5ff00ac36f7 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:04.368159 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 238/484] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:04.379356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20dd13-54e7-409c-8001-b68bcfe37a3a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:04.379808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20dd13-54e7-409c-8001-b68bcfe37a3a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:04.380170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20dd13-54e7-409c-8001-b68bcfe37a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:04.418897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2d20dd13-54e7-409c-8001-b68bcfe37a3a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:04.420090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2d20dd13-54e7-409c-8001-b68bcfe37a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:04.464745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7b4a3-6946-4661-9226-c5b03c7ebc0e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:04.525042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502eaa8-e3bd-4b7f-b016-06ba235285f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:04.529035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502eaa8-e3bd-4b7f-b016-06ba235285f4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:04.529502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502eaa8-e3bd-4b7f-b016-06ba235285f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:04.530002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502eaa8-e3bd-4b7f-b016-06ba235285f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:04.530991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502eaa8-e3bd-4b7f-b016-06ba235285f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:04.531471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502eaa8-e3bd-4b7f-b016-06ba235285f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:04.533038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502eaa8-e3bd-4b7f-b016-06ba235285f4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:04.554548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 247/485] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:04.557233 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 239/486] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:04.563711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502eaa8-e3bd-4b7f-b016-06ba235285f4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:04.600019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20dd13-54e7-409c-8001-b68bcfe37a3a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:04.621503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:04.727936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:04.739950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502eaa8-e3bd-4b7f-b016-06ba235285f4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:04.763944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20dd13-54e7-409c-8001-b68bcfe37a3a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:04.775671 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54bdda-1b44-4d30-bf1c-a1e84b53c3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:04.776545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54bdda-1b44-4d30-bf1c-a1e84b53c3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:04.777112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54bdda-1b44-4d30-bf1c-a1e84b53c3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:04.777112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54bdda-1b44-4d30-bf1c-a1e84b53c3f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:04.788761 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54bdda-1b44-4d30-bf1c-a1e84b53c3f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:04.790372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54bdda-1b44-4d30-bf1c-a1e84b53c3f7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:04.804954 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cdafa-6dfa-4583-8c03-eaf521eae310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:04.806079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cdafa-6dfa-4583-8c03-eaf521eae310 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:04.806525 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cdafa-6dfa-4583-8c03-eaf521eae310 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:04.806957 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cdafa-6dfa-4583-8c03-eaf521eae310 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2a54bdda-1b44-4d30-bf1c-a1e84b53c3f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:04.809107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:04.809968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:04.810974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:04.810974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:04.810974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:04.810974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:04.811475 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 248/487] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:04.812728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895cae2-d80d-4491-83e7-30f93cb6fc96 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:04.813032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895cae2-d80d-4491-83e7-30f93cb6fc96 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:04.813294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895cae2-d80d-4491-83e7-30f93cb6fc96 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:04.817970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 249/488] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:04.818969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895cae2-d80d-4491-83e7-30f93cb6fc96 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:04.825523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732cdafa-6dfa-4583-8c03-eaf521eae310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-732cdafa-6dfa-4583-8c03-eaf521eae310 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:04.839750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:04.840723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:04.841520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:04.844033 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 240/489] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:04.845464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efe91d-6c68-4128-8a1b-adfa26a779d2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:04.845945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efe91d-6c68-4128-8a1b-adfa26a779d2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:04.846606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efe91d-6c68-4128-8a1b-adfa26a779d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:04.866365 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-30efe91d-6c68-4128-8a1b-adfa26a779d2 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:04.867787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-30efe91d-6c68-4128-8a1b-adfa26a779d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:04.912438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:04.933172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895cae2-d80d-4491-83e7-30f93cb6fc96 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:04.944709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 241/490] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:04.946409 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efe91d-6c68-4128-8a1b-adfa26a779d2 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:04.990285 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825e5e9-5779-4438-bf16-89971c022b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:04.990957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825e5e9-5779-4438-bf16-89971c022b90 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:04.991240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825e5e9-5779-4438-bf16-89971c022b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:04.991515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825e5e9-5779-4438-bf16-89971c022b90 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:04.992243 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825e5e9-5779-4438-bf16-89971c022b90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:04.992735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825e5e9-5779-4438-bf16-89971c022b90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:04.994160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825e5e9-5779-4438-bf16-89971c022b90 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:05.010704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 250/491] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:05.010704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15a0d8-bc34-4097-89fc-98323302b07e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.010704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15a0d8-bc34-4097-89fc-98323302b07e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.010704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15a0d8-bc34-4097-89fc-98323302b07e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.010704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 251/492] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:05.010704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15a0d8-bc34-4097-89fc-98323302b07e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:05.018824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:05.059666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efe91d-6c68-4128-8a1b-adfa26a779d2 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:05.094086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5c9d4-3aec-466c-af6e-d83a48e33e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:05.094753 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c9d4-3aec-466c-af6e-d83a48e33e67 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.095034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c9d4-3aec-466c-af6e-d83a48e33e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.095361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c9d4-3aec-466c-af6e-d83a48e33e67 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.096242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5c9d4-3aec-466c-af6e-d83a48e33e67 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:05.096555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5c9d4-3aec-466c-af6e-d83a48e33e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:05.098082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5c9d4-3aec-466c-af6e-d83a48e33e67 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:05.111476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 242/493] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:05.112546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5c9d4-3aec-466c-af6e-d83a48e33e67 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:05.135023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15a0d8-bc34-4097-89fc-98323302b07e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:05.175452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5b4f6-fe1b-4fb9-8f61-db21c6b6fe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:05.176482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5b4f6-fe1b-4fb9-8f61-db21c6b6fe60 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.176728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5b4f6-fe1b-4fb9-8f61-db21c6b6fe60 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.177003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5b4f6-fe1b-4fb9-8f61-db21c6b6fe60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.185313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:05.188312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5b4f6-fe1b-4fb9-8f61-db21c6b6fe60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-38a5b4f6-fe1b-4fb9-8f61-db21c6b6fe60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:05.196900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:05.197790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:05.198666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 252/494] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:05.199410 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f271-c67a-497b-8dab-cf885ef668bc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.200636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f271-c67a-497b-8dab-cf885ef668bc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.200636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f271-c67a-497b-8dab-cf885ef668bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.217439 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1750f271-c67a-497b-8dab-cf885ef668bc None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:05.218818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1750f271-c67a-497b-8dab-cf885ef668bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:05.229839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5c9d4-3aec-466c-af6e-d83a48e33e67 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:05.254777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:05.266782 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0c82a-e955-40ed-ae01-cd2b5d2d1a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:05.267548 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0c82a-e955-40ed-ae01-cd2b5d2d1a21 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.267830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0c82a-e955-40ed-ae01-cd2b5d2d1a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.268317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0c82a-e955-40ed-ae01-cd2b5d2d1a21 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.276401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0c82a-e955-40ed-ae01-cd2b5d2d1a21 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:05.277854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0c82a-e955-40ed-ae01-cd2b5d2d1a21 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:05.289328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 253/495] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d6d0c82a-e955-40ed-ae01-cd2b5d2d1a21 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:05.293389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:05.294504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:05.295424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:05.295424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:05.295424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:05.295424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:05.295424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:05.295424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:05.295424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:05.295424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:05.295424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:05.295424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:05.296955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750f271-c67a-497b-8dab-cf885ef668bc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:05.299257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 243/496] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 4 09:02:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:05.300085 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0c82a-e955-40ed-ae01-cd2b5d2d1a21 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:05.409255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0c82a-e955-40ed-ae01-cd2b5d2d1a21 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:05.414565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750f271-c67a-497b-8dab-cf885ef668bc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:05.444628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ecf7d-f9ef-4ce2-b61b-3c66f66a8000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:05.445294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ecf7d-f9ef-4ce2-b61b-3c66f66a8000 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.445574 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ecf7d-f9ef-4ce2-b61b-3c66f66a8000 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.445844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ecf7d-f9ef-4ce2-b61b-3c66f66a8000 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.446625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ecf7d-f9ef-4ce2-b61b-3c66f66a8000 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:05.446929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ecf7d-f9ef-4ce2-b61b-3c66f66a8000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:05.448312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ecf7d-f9ef-4ce2-b61b-3c66f66a8000 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:05.449310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:05.455814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416b604-f2c0-4330-8434-1e7240d15acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:05.456608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416b604-f2c0-4330-8434-1e7240d15acc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.456977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 244/497] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:05.457782 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ecf7d-f9ef-4ce2-b61b-3c66f66a8000 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:05.461519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416b604-f2c0-4330-8434-1e7240d15acc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.461828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416b604-f2c0-4330-8434-1e7240d15acc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.473865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416b604-f2c0-4330-8434-1e7240d15acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:05.475716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416b604-f2c0-4330-8434-1e7240d15acc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b416b604-f2c0-4330-8434-1e7240d15acc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:05.495496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:05.496420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:05.497276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:05.497276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:05.497276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:05.497276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:05.501328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 254/498] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:05.502399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416b604-f2c0-4330-8434-1e7240d15acc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:05.506140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:05.563883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ecf7d-f9ef-4ce2-b61b-3c66f66a8000 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:05.598918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5666b0-f097-4119-bb8c-8de029869cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:05.599593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5666b0-f097-4119-bb8c-8de029869cdb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.599878 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5666b0-f097-4119-bb8c-8de029869cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.600153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5666b0-f097-4119-bb8c-8de029869cdb None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.600998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5666b0-f097-4119-bb8c-8de029869cdb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:05.601311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5666b0-f097-4119-bb8c-8de029869cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:05.602787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5666b0-f097-4119-bb8c-8de029869cdb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:05.607732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416b604-f2c0-4330-8434-1e7240d15acc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:05.640593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 245/499] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 4 09:02:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:05.641433 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5666b0-f097-4119-bb8c-8de029869cdb None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:05.647896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5f10f-6b58-4aca-abb9-93b7b5c9631c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:05.649009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5f10f-6b58-4aca-abb9-93b7b5c9631c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.649311 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5f10f-6b58-4aca-abb9-93b7b5c9631c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.649581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5f10f-6b58-4aca-abb9-93b7b5c9631c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.661100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5f10f-6b58-4aca-abb9-93b7b5c9631c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f7d5f10f-6b58-4aca-abb9-93b7b5c9631c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:05.669465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:05.670422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:05.671249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 255/500] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:05.671924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5f10f-6b58-4aca-abb9-93b7b5c9631c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:05.718040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:05.753567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5666b0-f097-4119-bb8c-8de029869cdb None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:05.756069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:05.779533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5f10f-6b58-4aca-abb9-93b7b5c9631c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:05.788264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f345f-8887-4e97-b31e-39114038647c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:05.789076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f345f-8887-4e97-b31e-39114038647c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.789358 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f345f-8887-4e97-b31e-39114038647c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.789633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f345f-8887-4e97-b31e-39114038647c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.800222 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f345f-8887-4e97-b31e-39114038647c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:05.802041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f345f-8887-4e97-b31e-39114038647c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:05.814843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b908861-1ae0-4034-ad35-410443621694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:05.815511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b908861-1ae0-4034-ad35-410443621694 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.815799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b908861-1ae0-4034-ad35-410443621694 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.816080 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b908861-1ae0-4034-ad35-410443621694 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.817395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b908861-1ae0-4034-ad35-410443621694 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:05.817712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b908861-1ae0-4034-ad35-410443621694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:05.819136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b908861-1ae0-4034-ad35-410443621694 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ca9f345f-8887-4e97-b31e-39114038647c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:05.820193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:05.821006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:05.821917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:05.821917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:05.821917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:05.821917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:05.822387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 246/501] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:05.823266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f345f-8887-4e97-b31e-39114038647c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:05.835982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 256/502] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:05.835982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b908861-1ae0-4034-ad35-410443621694 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:05.925659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f345f-8887-4e97-b31e-39114038647c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:05.938064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b908861-1ae0-4034-ad35-410443621694 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:05.939562 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:05.965578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cab84-5523-46e2-a2c3-e45a520bd306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:05.966257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cab84-5523-46e2-a2c3-e45a520bd306 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.966511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cab84-5523-46e2-a2c3-e45a520bd306 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.966809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cab84-5523-46e2-a2c3-e45a520bd306 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.977616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54b15a-b749-4109-9386-68344939807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:05.978305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54b15a-b749-4109-9386-68344939807d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.978588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54b15a-b749-4109-9386-68344939807d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.978861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54b15a-b749-4109-9386-68344939807d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.979883 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54b15a-b749-4109-9386-68344939807d None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:05.980195 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54b15a-b749-4109-9386-68344939807d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:05.981557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54b15a-b749-4109-9386-68344939807d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:05.989566 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 257/503] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:05.990862 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c1918-542e-490f-83bf-e4c21f75c8c8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:05.991398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c1918-542e-490f-83bf-e4c21f75c8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:05.991398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c1918-542e-490f-83bf-e4c21f75c8c8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:05.992873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 258/504] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:05.993038 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:05.993571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c1918-542e-490f-83bf-e4c21f75c8c8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:06.040784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cab84-5523-46e2-a2c3-e45a520bd306 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:06.042297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cab84-5523-46e2-a2c3-e45a520bd306 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:06.095255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c1918-542e-490f-83bf-e4c21f75c8c8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:06.128704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087c4b6-de66-4446-be54-26046a2fb9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:06.129489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087c4b6-de66-4446-be54-26046a2fb9bc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:06.129765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087c4b6-de66-4446-be54-26046a2fb9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:06.130076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087c4b6-de66-4446-be54-26046a2fb9bc None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:06.130823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087c4b6-de66-4446-be54-26046a2fb9bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:06.131134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087c4b6-de66-4446-be54-26046a2fb9bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:06.132529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087c4b6-de66-4446-be54-26046a2fb9bc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:06.142392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 247/505] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:02:06.143232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cab84-5523-46e2-a2c3-e45a520bd306 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:06.148950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 259/506] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:06.148950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087c4b6-de66-4446-be54-26046a2fb9bc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:06.174502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:06.201105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:06.246471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cab84-5523-46e2-a2c3-e45a520bd306 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:06.247389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087c4b6-de66-4446-be54-26046a2fb9bc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:06.284227 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a288ef-c986-4a0b-ac43-ddb5bbd00ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:06.284522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113d01e-33df-44ee-9f0e-fea004e8600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:06.285016 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a288ef-c986-4a0b-ac43-ddb5bbd00ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:06.285313 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a288ef-c986-4a0b-ac43-ddb5bbd00ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:06.285593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a288ef-c986-4a0b-ac43-ddb5bbd00ab7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:06.285845 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113d01e-33df-44ee-9f0e-fea004e8600e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:06.286168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113d01e-33df-44ee-9f0e-fea004e8600e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:06.286453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113d01e-33df-44ee-9f0e-fea004e8600e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:06.292833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a288ef-c986-4a0b-ac43-ddb5bbd00ab7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:06.294319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a288ef-c986-4a0b-ac43-ddb5bbd00ab7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:06.296827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113d01e-33df-44ee-9f0e-fea004e8600e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:06.298387 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113d01e-33df-44ee-9f0e-fea004e8600e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-55a288ef-c986-4a0b-ac43-ddb5bbd00ab7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:06.308944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:06.309846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:06.310833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:06.310833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:06.310833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:06.310833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:06.310833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:06.310833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:06.310833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:06.310833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:06.310833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:02:06.310833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:06.310833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 248/507] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:06.311798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1353a-9a81-4fd8-b87b-8a5c56e4ac0e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:06.311798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1353a-9a81-4fd8-b87b-8a5c56e4ac0e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:06.311798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1353a-9a81-4fd8-b87b-8a5c56e4ac0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-3113d01e-33df-44ee-9f0e-fea004e8600e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:06.315805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:06.316654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:06.317482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:06.317482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:06.317482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:06.317482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:06.317482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 260/508] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:06.317759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113d01e-33df-44ee-9f0e-fea004e8600e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:06.330301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c4d1353a-9a81-4fd8-b87b-8a5c56e4ac0e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:06.331483 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c4d1353a-9a81-4fd8-b87b-8a5c56e4ac0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:06.397425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:06.397761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 249/509] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:06.398618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1353a-9a81-4fd8-b87b-8a5c56e4ac0e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:06.428280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113d01e-33df-44ee-9f0e-fea004e8600e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:06.444555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:06.463322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b666275-0723-4208-9699-fe93e0817567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:06.464045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b666275-0723-4208-9699-fe93e0817567 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:06.464325 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b666275-0723-4208-9699-fe93e0817567 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:06.464604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b666275-0723-4208-9699-fe93e0817567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:06.474645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b666275-0723-4208-9699-fe93e0817567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:06.476144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b666275-0723-4208-9699-fe93e0817567 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2b666275-0723-4208-9699-fe93e0817567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:06.493147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:06.494072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:06.494941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:06.494941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:06.494941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:06.494941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:06.494941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 261/510] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:06.495215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b666275-0723-4208-9699-fe93e0817567 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:06.502658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1353a-9a81-4fd8-b87b-8a5c56e4ac0e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:06.537143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf83874-16a6-4bee-b43a-404263a0b5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:06.538012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf83874-16a6-4bee-b43a-404263a0b5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:06.538291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf83874-16a6-4bee-b43a-404263a0b5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:06.538569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf83874-16a6-4bee-b43a-404263a0b5e0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:06.545608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf83874-16a6-4bee-b43a-404263a0b5e0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:06.547108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf83874-16a6-4bee-b43a-404263a0b5e0 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-daf83874-16a6-4bee-b43a-404263a0b5e0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:06.561493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:06.562468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:06.563363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:06.563363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:06.563363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:06.563363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:06.563363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:06.563363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:06.563363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:06.563363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:06.563363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:02:06.563363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:06.563363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 250/511] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:06.563948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf83874-16a6-4bee-b43a-404263a0b5e0 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:06.605996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b666275-0723-4208-9699-fe93e0817567 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:06.635034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:06.644004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca88a-3f19-4736-ba38-6642e44a104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:06.644684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca88a-3f19-4736-ba38-6642e44a104a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:06.644967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca88a-3f19-4736-ba38-6642e44a104a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:06.645244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca88a-3f19-4736-ba38-6642e44a104a None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:06.646124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ca88a-3f19-4736-ba38-6642e44a104a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:06.646442 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ca88a-3f19-4736-ba38-6642e44a104a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:06.647889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ca88a-3f19-4736-ba38-6642e44a104a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:06.678420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf83874-16a6-4bee-b43a-404263a0b5e0 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:06.684360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 262/512] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 4 09:02:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:06.685172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca88a-3f19-4736-ba38-6642e44a104a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:06.685804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:06.717377 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61934f6e-d76e-486b-a103-efcee538df19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:06.717995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61934f6e-d76e-486b-a103-efcee538df19 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:06.718279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61934f6e-d76e-486b-a103-efcee538df19 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:06.718559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61934f6e-d76e-486b-a103-efcee538df19 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:06.719287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61934f6e-d76e-486b-a103-efcee538df19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:06.719594 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61934f6e-d76e-486b-a103-efcee538df19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:06.720984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61934f6e-d76e-486b-a103-efcee538df19 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:06.732164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 251/513] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:06.732945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61934f6e-d76e-486b-a103-efcee538df19 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:06.782756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca88a-3f19-4736-ba38-6642e44a104a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:06.817844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c598f9-aa89-4f9e-9d56-fe45e85119d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:06.818656 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c598f9-aa89-4f9e-9d56-fe45e85119d0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:06.818941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c598f9-aa89-4f9e-9d56-fe45e85119d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:06.819234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c598f9-aa89-4f9e-9d56-fe45e85119d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:06.829087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c598f9-aa89-4f9e-9d56-fe45e85119d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:06.830602 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c598f9-aa89-4f9e-9d56-fe45e85119d0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:06.833732 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61934f6e-d76e-486b-a103-efcee538df19 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-00c598f9-aa89-4f9e-9d56-fe45e85119d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:06.848191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:06.849063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:06.849947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:06.849947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:06.849947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:06.849947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:06.849947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 263/514] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:06.850510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c598f9-aa89-4f9e-9d56-fe45e85119d0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:06.871193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f50db-542c-4360-894c-23c6752a592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:06.871852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f50db-542c-4360-894c-23c6752a592f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:06.872135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f50db-542c-4360-894c-23c6752a592f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:06.872409 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f50db-542c-4360-894c-23c6752a592f None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:06.873135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f50db-542c-4360-894c-23c6752a592f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:06.873436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f50db-542c-4360-894c-23c6752a592f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:06.874839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f50db-542c-4360-894c-23c6752a592f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:06.886950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 252/515] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:06.887740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f50db-542c-4360-894c-23c6752a592f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:06.890748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:06.915231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:06.952054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c598f9-aa89-4f9e-9d56-fe45e85119d0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:06.986691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a456668-7172-4ca7-af3b-7193442321dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:06.987091 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f50db-542c-4360-894c-23c6752a592f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:06.987548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a456668-7172-4ca7-af3b-7193442321dc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:06.987548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a456668-7172-4ca7-af3b-7193442321dc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:06.987824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a456668-7172-4ca7-af3b-7193442321dc None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:06.988858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a456668-7172-4ca7-af3b-7193442321dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:06.989137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a456668-7172-4ca7-af3b-7193442321dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:06.990666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a456668-7172-4ca7-af3b-7193442321dc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:07.008096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 264/516] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 4 09:02:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:07.008976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a456668-7172-4ca7-af3b-7193442321dc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:07.023556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d44bb-ae92-4910-af1a-eb5391ffdd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:07.024213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d44bb-ae92-4910-af1a-eb5391ffdd1a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:07.024496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d44bb-ae92-4910-af1a-eb5391ffdd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:07.024768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d44bb-ae92-4910-af1a-eb5391ffdd1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:07.091762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d44bb-ae92-4910-af1a-eb5391ffdd1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:07.093222 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d44bb-ae92-4910-af1a-eb5391ffdd1a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:07.100081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:07.106104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a456668-7172-4ca7-af3b-7193442321dc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:07.120580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:07.141565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555fa68-a368-40b5-80ad-ee6ab0e96b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:07.142404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555fa68-a368-40b5-80ad-ee6ab0e96b2f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:07.142687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555fa68-a368-40b5-80ad-ee6ab0e96b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:07.142955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555fa68-a368-40b5-80ad-ee6ab0e96b2f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:07.150271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555fa68-a368-40b5-80ad-ee6ab0e96b2f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:07.151716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555fa68-a368-40b5-80ad-ee6ab0e96b2f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f555fa68-a368-40b5-80ad-ee6ab0e96b2f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:07.169669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:07.170669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:07.171577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:07.171577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:07.171577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:07.171577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:07.171577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:07.171577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:07.171577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:07.171577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:07.171577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:02:07.171577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:07.171577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 265/517] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:07.172175 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555fa68-a368-40b5-80ad-ee6ab0e96b2f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:07.199284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 253/518] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:02:07.200137 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d44bb-ae92-4910-af1a-eb5391ffdd1a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:07.268320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555fa68-a368-40b5-80ad-ee6ab0e96b2f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:07.297178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d44bb-ae92-4910-af1a-eb5391ffdd1a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:07.301844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea0d61-5ca0-4d01-8146-9310da72a080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:07.302537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea0d61-5ca0-4d01-8146-9310da72a080 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:07.302820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea0d61-5ca0-4d01-8146-9310da72a080 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:07.303098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea0d61-5ca0-4d01-8146-9310da72a080 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:07.303823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea0d61-5ca0-4d01-8146-9310da72a080 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:07.304130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea0d61-5ca0-4d01-8146-9310da72a080 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:07.305522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea0d61-5ca0-4d01-8146-9310da72a080 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:07.316809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 266/519] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:07.317599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea0d61-5ca0-4d01-8146-9310da72a080 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:07.323786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:07.333402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b7701-e5c5-4136-904c-0800efdfc7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:07.334295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b7701-e5c5-4136-904c-0800efdfc7cb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:07.334576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b7701-e5c5-4136-904c-0800efdfc7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:07.334850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b7701-e5c5-4136-904c-0800efdfc7cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:07.335788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:07.342160 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b7701-e5c5-4136-904c-0800efdfc7cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:07.343618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b7701-e5c5-4136-904c-0800efdfc7cb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-235b7701-e5c5-4136-904c-0800efdfc7cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:07.358113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:07.359132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:07.359987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:07.359987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:07.359987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:07.359987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:07.359987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:07.359987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:07.359987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:07.359987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:07.362023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 254/520] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:07.363056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b7701-e5c5-4136-904c-0800efdfc7cb None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:07.416218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea0d61-5ca0-4d01-8146-9310da72a080 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:07.450433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce061867-63e7-4050-b525-f62d827f4e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:07.451082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce061867-63e7-4050-b525-f62d827f4e2f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:07.451369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce061867-63e7-4050-b525-f62d827f4e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:07.451647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce061867-63e7-4050-b525-f62d827f4e2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:07.469223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b7701-e5c5-4136-904c-0800efdfc7cb None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:07.512251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e98746-e6e3-47cf-bdde-8dbf68eab86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:07.513195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e98746-e6e3-47cf-bdde-8dbf68eab86f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:07.513668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e98746-e6e3-47cf-bdde-8dbf68eab86f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:07.514040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e98746-e6e3-47cf-bdde-8dbf68eab86f None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:07.515338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e98746-e6e3-47cf-bdde-8dbf68eab86f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:07.515804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e98746-e6e3-47cf-bdde-8dbf68eab86f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:07.517733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e98746-e6e3-47cf-bdde-8dbf68eab86f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:07.519350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce061867-63e7-4050-b525-f62d827f4e2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:07.520967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce061867-63e7-4050-b525-f62d827f4e2f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:07.538198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:07.538600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 255/521] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jul 4 09:02:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:07.539421 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e98746-e6e3-47cf-bdde-8dbf68eab86f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:07.587909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:07.624894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 267/522] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:02:07.630208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce061867-63e7-4050-b525-f62d827f4e2f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:07.642599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e98746-e6e3-47cf-bdde-8dbf68eab86f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:07.676131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26675d-a2ea-4c13-a298-631fdc5cd92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:07.676921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26675d-a2ea-4c13-a298-631fdc5cd92e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:07.677200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26675d-a2ea-4c13-a298-631fdc5cd92e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:07.677480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26675d-a2ea-4c13-a298-631fdc5cd92e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:07.687383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26675d-a2ea-4c13-a298-631fdc5cd92e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:07.688847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26675d-a2ea-4c13-a298-631fdc5cd92e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-4a26675d-a2ea-4c13-a298-631fdc5cd92e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:07.707194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:07.708263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:07.709113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:07.709113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:07.709113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:07.709113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:07.709113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 256/523] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:07.709113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26675d-a2ea-4c13-a298-631fdc5cd92e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:07.725946 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce061867-63e7-4050-b525-f62d827f4e2f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:07.742307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:07.759953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bc076-841a-4449-9d40-180cffd6c3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:07.760615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bc076-841a-4449-9d40-180cffd6c3f4 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:07.760895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bc076-841a-4449-9d40-180cffd6c3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:07.761165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bc076-841a-4449-9d40-180cffd6c3f4 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:07.761867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bc076-841a-4449-9d40-180cffd6c3f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:07.763338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bc076-841a-4449-9d40-180cffd6c3f4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:07.783123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 268/524] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:02:07 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:07.783851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bc076-841a-4449-9d40-180cffd6c3f4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:07.812586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:07.821406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26675d-a2ea-4c13-a298-631fdc5cd92e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:07.865054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f54656-45e9-43e1-b8c4-28b976fd71b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:07.865959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f54656-45e9-43e1-b8c4-28b976fd71b7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:07.866340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f54656-45e9-43e1-b8c4-28b976fd71b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:07.866710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f54656-45e9-43e1-b8c4-28b976fd71b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:07.867643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f54656-45e9-43e1-b8c4-28b976fd71b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:07.868077 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f54656-45e9-43e1-b8c4-28b976fd71b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:07.869705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f54656-45e9-43e1-b8c4-28b976fd71b7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:07.894183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 257/525] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:07.894183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f54656-45e9-43e1-b8c4-28b976fd71b7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:07.916207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bc076-841a-4449-9d40-180cffd6c3f4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:07.960399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846200a-0b63-48a8-88ff-e655b3587a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:07.961424 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846200a-0b63-48a8-88ff-e655b3587a33 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:07.961539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846200a-0b63-48a8-88ff-e655b3587a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:07.961730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846200a-0b63-48a8-88ff-e655b3587a33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:08.005677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f54656-45e9-43e1-b8c4-28b976fd71b7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:08.008228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:08.039294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846200a-0b63-48a8-88ff-e655b3587a33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:08.040669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846200a-0b63-48a8-88ff-e655b3587a33 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:08.043145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfdfd3-c8e5-4639-9226-15d8b6b830c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:08.044714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfdfd3-c8e5-4639-9226-15d8b6b830c7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:08.045002 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfdfd3-c8e5-4639-9226-15d8b6b830c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:08.045279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfdfd3-c8e5-4639-9226-15d8b6b830c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:08.058331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfdfd3-c8e5-4639-9226-15d8b6b830c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:08.060081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfdfd3-c8e5-4639-9226-15d8b6b830c7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-3ebfdfd3-c8e5-4639-9226-15d8b6b830c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:08.082697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:08.083883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:08.085263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:08.085263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:08.085263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:08.085263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:08.085263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 258/526] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:08.085263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfdfd3-c8e5-4639-9226-15d8b6b830c7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:08.097440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:08.148776 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 269/527] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:02:08.149651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846200a-0b63-48a8-88ff-e655b3587a33 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:08.206648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfdfd3-c8e5-4639-9226-15d8b6b830c7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:08.227540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:08.244426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed432bf-d84a-469d-abf9-4fcd7c5eb895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:08.245100 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed432bf-d84a-469d-abf9-4fcd7c5eb895 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:08.245383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed432bf-d84a-469d-abf9-4fcd7c5eb895 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:08.245655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed432bf-d84a-469d-abf9-4fcd7c5eb895 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:08.246840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed432bf-d84a-469d-abf9-4fcd7c5eb895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:08.247158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed432bf-d84a-469d-abf9-4fcd7c5eb895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:08.248983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed432bf-d84a-469d-abf9-4fcd7c5eb895 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:08.259735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846200a-0b63-48a8-88ff-e655b3587a33 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:08.266626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 259/528] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:08.266626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed432bf-d84a-469d-abf9-4fcd7c5eb895 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:08.303724 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fd7e3-9cfc-48d2-8c26-d9b620026700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:08.304441 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fd7e3-9cfc-48d2-8c26-d9b620026700 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:08.304817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fd7e3-9cfc-48d2-8c26-d9b620026700 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:08.305204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fd7e3-9cfc-48d2-8c26-d9b620026700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:08.306451 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453fd7e3-9cfc-48d2-8c26-d9b620026700 None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=739f3b5eaef542c29a0cf0b475c0d965) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:08.306868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453fd7e3-9cfc-48d2-8c26-d9b620026700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:08.308404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453fd7e3-9cfc-48d2-8c26-d9b620026700 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:08.314075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 270/529] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:08 2022] GET /identity/v3/role_assignments?user.id=739f3b5eaef542c29a0cf0b475c0d965&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:08.323283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fd7e3-9cfc-48d2-8c26-d9b620026700 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:08.339522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:08.389081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed432bf-d84a-469d-abf9-4fcd7c5eb895 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:08.436370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5abd57-1b84-448a-baa3-55d493cf61d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:08.437239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5abd57-1b84-448a-baa3-55d493cf61d3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:08.437622 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5abd57-1b84-448a-baa3-55d493cf61d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:08.438030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5abd57-1b84-448a-baa3-55d493cf61d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:08.438901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5abd57-1b84-448a-baa3-55d493cf61d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:08.439310 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5abd57-1b84-448a-baa3-55d493cf61d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:08.441479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5abd57-1b84-448a-baa3-55d493cf61d3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:08.454232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fd7e3-9cfc-48d2-8c26-d9b620026700 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:08.459668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 260/530] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:08.460778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5abd57-1b84-448a-baa3-55d493cf61d3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:08.501690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a6f4a-8bc2-48d9-9d90-c90ec38cb10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:08.503150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a6f4a-8bc2-48d9-9d90-c90ec38cb10c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:08.503430 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a6f4a-8bc2-48d9-9d90-c90ec38cb10c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:08.503701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a6f4a-8bc2-48d9-9d90-c90ec38cb10c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:08.508028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:08.516532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a6f4a-8bc2-48d9-9d90-c90ec38cb10c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:08.518875 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a6f4a-8bc2-48d9-9d90-c90ec38cb10c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a71a6f4a-8bc2-48d9-9d90-c90ec38cb10c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:08.548426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:08.549541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:08.550626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:08.550626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:08.550626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:08.550626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:08.554188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 271/531] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:08.555585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a6f4a-8bc2-48d9-9d90-c90ec38cb10c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:08.589279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5abd57-1b84-448a-baa3-55d493cf61d3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:08.631325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:08.642018 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ceb48-7c1a-47f4-93e0-08d4056706fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:08.643101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ceb48-7c1a-47f4-93e0-08d4056706fd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:08.643557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ceb48-7c1a-47f4-93e0-08d4056706fd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:08.644433 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ceb48-7c1a-47f4-93e0-08d4056706fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:08.652950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ceb48-7c1a-47f4-93e0-08d4056706fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:08.654744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ceb48-7c1a-47f4-93e0-08d4056706fd None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-209ceb48-7c1a-47f4-93e0-08d4056706fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:08.669116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:08.670025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:08.675949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:08.675949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:08.675949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:08.675949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:08.675949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:08.675949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:08.675949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:08.675949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:08.675949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 261/532] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:08.675949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ceb48-7c1a-47f4-93e0-08d4056706fd None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:08.676491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a6f4a-8bc2-48d9-9d90-c90ec38cb10c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:08.726377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36180691-9046-4284-9db5-cccb9104bf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:08.727944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36180691-9046-4284-9db5-cccb9104bf05 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:08.728423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36180691-9046-4284-9db5-cccb9104bf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:08.728889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36180691-9046-4284-9db5-cccb9104bf05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:08.738821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36180691-9046-4284-9db5-cccb9104bf05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:08.741521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36180691-9046-4284-9db5-cccb9104bf05 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-36180691-9046-4284-9db5-cccb9104bf05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:08.761636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:08.762587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:08.763458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:08.763458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:08.763458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:08.763458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:08.763458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:08.763458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:08.763458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:08.763458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:08.766005 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 272/533] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:08.767058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36180691-9046-4284-9db5-cccb9104bf05 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:08.789839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ceb48-7c1a-47f4-93e0-08d4056706fd None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:08.826346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408e70c-748d-492a-9d6f-9b363941ee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:08.827203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408e70c-748d-492a-9d6f-9b363941ee37 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:08.827630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408e70c-748d-492a-9d6f-9b363941ee37 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:08.828063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408e70c-748d-492a-9d6f-9b363941ee37 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:08.828929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408e70c-748d-492a-9d6f-9b363941ee37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:08.829327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408e70c-748d-492a-9d6f-9b363941ee37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:08.830876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408e70c-748d-492a-9d6f-9b363941ee37 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:08.837557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:08.844280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 262/534] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:08.849657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408e70c-748d-492a-9d6f-9b363941ee37 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:08.897854 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36180691-9046-4284-9db5-cccb9104bf05 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:08.938409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ab02d-a7a8-42ed-bbfb-f7fb1dcf8c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:08.939313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ab02d-a7a8-42ed-bbfb-f7fb1dcf8c7f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:08.939922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ab02d-a7a8-42ed-bbfb-f7fb1dcf8c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:08.940331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ab02d-a7a8-42ed-bbfb-f7fb1dcf8c7f None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:08.941872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ab02d-a7a8-42ed-bbfb-f7fb1dcf8c7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:08.942471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ab02d-a7a8-42ed-bbfb-f7fb1dcf8c7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:08.944217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ab02d-a7a8-42ed-bbfb-f7fb1dcf8c7f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:08.946245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:08.970352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 273/535] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:08.971646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ab02d-a7a8-42ed-bbfb-f7fb1dcf8c7f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:09.037077 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408e70c-748d-492a-9d6f-9b363941ee37 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:09.116289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081633e-c2aa-4414-abf0-683b1778ffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:09.116967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081633e-c2aa-4414-abf0-683b1778ffad None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:09.119003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081633e-c2aa-4414-abf0-683b1778ffad None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:09.119304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081633e-c2aa-4414-abf0-683b1778ffad None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:09.122021 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6081633e-c2aa-4414-abf0-683b1778ffad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:09.122347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6081633e-c2aa-4414-abf0-683b1778ffad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:09.123483 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ab02d-a7a8-42ed-bbfb-f7fb1dcf8c7f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:09.130024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6081633e-c2aa-4414-abf0-683b1778ffad None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:09.138320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:09.163345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 263/536] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:09.165093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081633e-c2aa-4414-abf0-683b1778ffad None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:09.177557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ce619-38d5-4d1c-93aa-0a66d5df6956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:09.179137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ce619-38d5-4d1c-93aa-0a66d5df6956 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:09.180018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ce619-38d5-4d1c-93aa-0a66d5df6956 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:09.180493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ce619-38d5-4d1c-93aa-0a66d5df6956 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:09.191650 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ce619-38d5-4d1c-93aa-0a66d5df6956 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:09.194045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ce619-38d5-4d1c-93aa-0a66d5df6956 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f13ce619-38d5-4d1c-93aa-0a66d5df6956 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:09.215574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:09.216446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:09.217314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:09.217314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:09.217314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:09.217314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:09.217314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:09.217314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:09.217314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:09.217314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:09.224505 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 274/537] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:09.225585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ce619-38d5-4d1c-93aa-0a66d5df6956 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:09.339805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ce619-38d5-4d1c-93aa-0a66d5df6956 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:09.364757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081633e-c2aa-4414-abf0-683b1778ffad None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:09.383639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:09.409114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e7695-e466-45b5-a37b-739587a76e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:09.409781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e7695-e466-45b5-a37b-739587a76e1d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:09.410374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e7695-e466-45b5-a37b-739587a76e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:09.410480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e7695-e466-45b5-a37b-739587a76e1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:09.411284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e7695-e466-45b5-a37b-739587a76e1d None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=e9f34c35a9ba4376a9b55ec5e23cf35f) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:09.411597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e7695-e466-45b5-a37b-739587a76e1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:09.413062 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e7695-e466-45b5-a37b-739587a76e1d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:09.432465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 275/538] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:09 2022] GET /identity/v3/role_assignments?user.id=e9f34c35a9ba4376a9b55ec5e23cf35f&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:09.432465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e7695-e466-45b5-a37b-739587a76e1d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:09.439105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdc343-59c5-4aff-be57-41bb744edea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:09.442668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdc343-59c5-4aff-be57-41bb744edea2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:09.442668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdc343-59c5-4aff-be57-41bb744edea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:09.442668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdc343-59c5-4aff-be57-41bb744edea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:09.444763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdc343-59c5-4aff-be57-41bb744edea2 None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=107a90e384504e8d80264d6bf20f1d9b) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:09.451673 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdc343-59c5-4aff-be57-41bb744edea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:09.453298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdc343-59c5-4aff-be57-41bb744edea2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:09.465752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 264/539] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:09 2022] GET /identity/v3/role_assignments?user.id=107a90e384504e8d80264d6bf20f1d9b&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:09.491265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:09.564499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e7695-e466-45b5-a37b-739587a76e1d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:09.601562 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e485a6e-2001-428a-81b6-8de89814ec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:09.602275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e485a6e-2001-428a-81b6-8de89814ec6e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:09.602557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e485a6e-2001-428a-81b6-8de89814ec6e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:09.602832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e485a6e-2001-428a-81b6-8de89814ec6e None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:09.603648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e485a6e-2001-428a-81b6-8de89814ec6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:09.603959 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e485a6e-2001-428a-81b6-8de89814ec6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:09.605430 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e485a6e-2001-428a-81b6-8de89814ec6e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:09.625743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 276/540] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:09.627538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdc343-59c5-4aff-be57-41bb744edea2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:09.750496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdc343-59c5-4aff-be57-41bb744edea2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:09.786648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4adbdf2-11c4-45a7-8566-34b658d5203f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:09.787312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adbdf2-11c4-45a7-8566-34b658d5203f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:09.787597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adbdf2-11c4-45a7-8566-34b658d5203f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:09.787873 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adbdf2-11c4-45a7-8566-34b658d5203f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:09.788710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4adbdf2-11c4-45a7-8566-34b658d5203f None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=98649aa38f764af0b41ff5594b94a989) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:09.789019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4adbdf2-11c4-45a7-8566-34b658d5203f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:09.790492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4adbdf2-11c4-45a7-8566-34b658d5203f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:09.800094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 265/541] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:09 2022] GET /identity/v3/role_assignments?user.id=98649aa38f764af0b41ff5594b94a989&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:09.880846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e94a8a-9d38-4ddc-9c43-81b5b00bf28d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:09.881171 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e94a8a-9d38-4ddc-9c43-81b5b00bf28d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:09.881408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e94a8a-9d38-4ddc-9c43-81b5b00bf28d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:09.883122 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 277/542] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:09.888570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30311fc-9275-4e43-8675-2357fe97b1f7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:09.888852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30311fc-9275-4e43-8675-2357fe97b1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:09.889133 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30311fc-9275-4e43-8675-2357fe97b1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:09.907417 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a30311fc-9275-4e43-8675-2357fe97b1f7 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:09.908624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a30311fc-9275-4e43-8675-2357fe97b1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:09.941771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:09.984332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 266/543] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:10.022892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e94a8a-9d38-4ddc-9c43-81b5b00bf28d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:10.122606 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:10.129746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e94a8a-9d38-4ddc-9c43-81b5b00bf28d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:10.166023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348294-41be-4cd9-b66e-916d84fd6096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:10.166835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348294-41be-4cd9-b66e-916d84fd6096 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:10.167116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348294-41be-4cd9-b66e-916d84fd6096 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:10.167387 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7348294-41be-4cd9-b66e-916d84fd6096 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:10.178507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7348294-41be-4cd9-b66e-916d84fd6096 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c7348294-41be-4cd9-b66e-916d84fd6096 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:10.186519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:10.187408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:10.188205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 278/544] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:10.195178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30311fc-9275-4e43-8675-2357fe97b1f7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:10.343830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30311fc-9275-4e43-8675-2357fe97b1f7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:10.396537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a8b2d-b286-438e-9afd-7283cda9e0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:10.397227 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a8b2d-b286-438e-9afd-7283cda9e0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:10.397514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a8b2d-b286-438e-9afd-7283cda9e0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:10.397796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a8b2d-b286-438e-9afd-7283cda9e0d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:10.398594 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a8b2d-b286-438e-9afd-7283cda9e0d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:10.398906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a8b2d-b286-438e-9afd-7283cda9e0d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:10.400301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a8b2d-b286-438e-9afd-7283cda9e0d6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:10.413749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 267/545] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:10.419348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348294-41be-4cd9-b66e-916d84fd6096 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:10.482501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185091c5-ae75-4bcb-b896-b457f92bbd04 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:10.482824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185091c5-ae75-4bcb-b896-b457f92bbd04 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:10.483073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185091c5-ae75-4bcb-b896-b457f92bbd04 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:10.484700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 268/546] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:10.489509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37a453-e86c-4461-b69b-927a0be41444 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:10.489800 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37a453-e86c-4461-b69b-927a0be41444 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:10.490105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37a453-e86c-4461-b69b-927a0be41444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:10.516539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-eb37a453-e86c-4461-b69b-927a0be41444 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:10.517723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-eb37a453-e86c-4461-b69b-927a0be41444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:10.544258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:10.610221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 269/547] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:10.613846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7348294-41be-4cd9-b66e-916d84fd6096 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:10.640365 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37a453-e86c-4461-b69b-927a0be41444 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:10.667151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858f355-ca9a-4da2-825b-7148173b9c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:10.667953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858f355-ca9a-4da2-825b-7148173b9c53 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:10.668241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858f355-ca9a-4da2-825b-7148173b9c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:10.668519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858f355-ca9a-4da2-825b-7148173b9c53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:10.680663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858f355-ca9a-4da2-825b-7148173b9c53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:10.682154 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858f355-ca9a-4da2-825b-7148173b9c53 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2858f355-ca9a-4da2-825b-7148173b9c53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:10.701198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:10.702075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:10.702952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:10.702952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:10.702952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:10.702952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:10.704393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 279/548] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:10.711073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858f355-ca9a-4da2-825b-7148173b9c53 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:10.738323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:10.834442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37a453-e86c-4461-b69b-927a0be41444 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:10.894516 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926badcd-dcec-4a84-aedb-ee2f7dfca431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:10.895934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926badcd-dcec-4a84-aedb-ee2f7dfca431 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:10.896221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926badcd-dcec-4a84-aedb-ee2f7dfca431 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:10.896498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926badcd-dcec-4a84-aedb-ee2f7dfca431 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:10.907271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926badcd-dcec-4a84-aedb-ee2f7dfca431 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:10.911596 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858f355-ca9a-4da2-825b-7148173b9c53 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-926badcd-dcec-4a84-aedb-ee2f7dfca431 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:10.915213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:10.916209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:10.917537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 270/549] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:10.922444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926badcd-dcec-4a84-aedb-ee2f7dfca431 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:10.950250 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:10.982559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b49708-0cb6-4dbb-a672-bc2accbc06bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:10.983265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b49708-0cb6-4dbb-a672-bc2accbc06bd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:10.983548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b49708-0cb6-4dbb-a672-bc2accbc06bd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:10.983822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b49708-0cb6-4dbb-a672-bc2accbc06bd None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:10.984564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b49708-0cb6-4dbb-a672-bc2accbc06bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:10.984873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b49708-0cb6-4dbb-a672-bc2accbc06bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:10.986301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b49708-0cb6-4dbb-a672-bc2accbc06bd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:11.007002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 280/550] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:11.010819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b49708-0cb6-4dbb-a672-bc2accbc06bd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:11.097611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926badcd-dcec-4a84-aedb-ee2f7dfca431 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:11.117327 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:11.154786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438be10a-7cd7-4bda-910d-f126cf2b1e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:11.155283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438be10a-7cd7-4bda-910d-f126cf2b1e4c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:11.155563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438be10a-7cd7-4bda-910d-f126cf2b1e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:11.155841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438be10a-7cd7-4bda-910d-f126cf2b1e4c None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:11.156612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438be10a-7cd7-4bda-910d-f126cf2b1e4c None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:11.156918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438be10a-7cd7-4bda-910d-f126cf2b1e4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:11.158322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438be10a-7cd7-4bda-910d-f126cf2b1e4c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:11.167482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 271/551] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:11.172831 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438be10a-7cd7-4bda-910d-f126cf2b1e4c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:11.189201 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b49708-0cb6-4dbb-a672-bc2accbc06bd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:11.256698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da1fb7-425d-4fdc-8604-672ca76176ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:11.257502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da1fb7-425d-4fdc-8604-672ca76176ec None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:11.257785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da1fb7-425d-4fdc-8604-672ca76176ec None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:11.258086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da1fb7-425d-4fdc-8604-672ca76176ec None admin] PATH_INFO: `/v3/users/swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:11.269554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da1fb7-425d-4fdc-8604-672ca76176ec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:11.271081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da1fb7-425d-4fdc-8604-672ca76176ec None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e9da1fb7-425d-4fdc-8604-672ca76176ec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:11.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:11.295672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:11.296692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:11.296692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:11.296692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:11.296692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:11.296692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:11.296692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:11.296692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:11.296692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:11.296692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:11.296692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:11.296692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 281/552] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:11.306875 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da1fb7-425d-4fdc-8604-672ca76176ec None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:11.318750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:11.340688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438be10a-7cd7-4bda-910d-f126cf2b1e4c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:11.381045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb64a9-209f-4192-88e9-ed37a96df1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:11.382477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb64a9-209f-4192-88e9-ed37a96df1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:11.383004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb64a9-209f-4192-88e9-ed37a96df1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:11.383472 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb64a9-209f-4192-88e9-ed37a96df1b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:11.396894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb64a9-209f-4192-88e9-ed37a96df1b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:11.398852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb64a9-209f-4192-88e9-ed37a96df1b8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-0deb64a9-209f-4192-88e9-ed37a96df1b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:11.419848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:11.420775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:11.421667 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:11.421667 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:11.421667 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:11.421667 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:11.424926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 272/553] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:11.431983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb64a9-209f-4192-88e9-ed37a96df1b8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:11.481917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da1fb7-425d-4fdc-8604-672ca76176ec None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:11.507256 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:11.518446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6fc74-7daf-4451-9cf2-36c0a758187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:11.519509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6fc74-7daf-4451-9cf2-36c0a758187e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:11.520076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6fc74-7daf-4451-9cf2-36c0a758187e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:11.520567 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6fc74-7daf-4451-9cf2-36c0a758187e None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:11.521637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6fc74-7daf-4451-9cf2-36c0a758187e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:11.522188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6fc74-7daf-4451-9cf2-36c0a758187e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:11.523846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6fc74-7daf-4451-9cf2-36c0a758187e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:11.542425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 282/554] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 4 09:02:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:11.550989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6fc74-7daf-4451-9cf2-36c0a758187e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:11.640044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb64a9-209f-4192-88e9-ed37a96df1b8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:11.661274 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6fc74-7daf-4451-9cf2-36c0a758187e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:11.702323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:11.711015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68d8ff-59fb-4142-810c-69bf24975278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:11.712187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68d8ff-59fb-4142-810c-69bf24975278 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:11.712187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68d8ff-59fb-4142-810c-69bf24975278 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:11.712398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68d8ff-59fb-4142-810c-69bf24975278 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:11.713129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68d8ff-59fb-4142-810c-69bf24975278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:11.713435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68d8ff-59fb-4142-810c-69bf24975278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:11.714893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68d8ff-59fb-4142-810c-69bf24975278 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:11.729222 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e265e1-eda9-416a-80c4-5556bc51cfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:11.729920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e265e1-eda9-416a-80c4-5556bc51cfbe None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:11.730206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e265e1-eda9-416a-80c4-5556bc51cfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:11.730478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e265e1-eda9-416a-80c4-5556bc51cfbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:11.745581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 273/555] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:11.752293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68d8ff-59fb-4142-810c-69bf24975278 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:11.802355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:11.822650 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e265e1-eda9-416a-80c4-5556bc51cfbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:11.824181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e265e1-eda9-416a-80c4-5556bc51cfbe None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:11.952735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 283/556] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:02:11.955040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acc224-fd14-4fd0-bb1b-c9d7a14f2fed None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:11.955040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acc224-fd14-4fd0-bb1b-c9d7a14f2fed None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:11.955040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acc224-fd14-4fd0-bb1b-c9d7a14f2fed None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:11.960929 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 284/557] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:11.960929 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acc224-fd14-4fd0-bb1b-c9d7a14f2fed None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:11.966456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68d8ff-59fb-4142-810c-69bf24975278 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:12.015987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53eb27-add1-4c28-a5c1-3d0e149f3f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:12.016449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53eb27-add1-4c28-a5c1-3d0e149f3f00 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.016649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53eb27-add1-4c28-a5c1-3d0e149f3f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.016927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53eb27-add1-4c28-a5c1-3d0e149f3f00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.028219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53eb27-add1-4c28-a5c1-3d0e149f3f00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:12.032242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53eb27-add1-4c28-a5c1-3d0e149f3f00 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-1e53eb27-add1-4c28-a5c1-3d0e149f3f00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:12.049798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:12.051386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:12.052339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:12.052339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:12.052339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:12.052339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:12.052339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:12.052339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:12.052339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:12.052339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:12.052339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:12.052339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:12.052339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 274/558] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 4 09:02:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:12.053482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90153960-17e4-48b7-9003-6eeaac365382 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.053482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90153960-17e4-48b7-9003-6eeaac365382 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.053651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90153960-17e4-48b7-9003-6eeaac365382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.078640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-90153960-17e4-48b7-9003-6eeaac365382 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:12.079832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-90153960-17e4-48b7-9003-6eeaac365382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:12.090108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:12.125197 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acc224-fd14-4fd0-bb1b-c9d7a14f2fed None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:12.147049 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:12.171513 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 275/559] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:12.171705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90153960-17e4-48b7-9003-6eeaac365382 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:12.185343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7122477-1aae-4db2-8e35-2ef76bc261be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:12.186688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7122477-1aae-4db2-8e35-2ef76bc261be None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.187646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7122477-1aae-4db2-8e35-2ef76bc261be None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.188132 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7122477-1aae-4db2-8e35-2ef76bc261be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.204073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7122477-1aae-4db2-8e35-2ef76bc261be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:12.205630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7122477-1aae-4db2-8e35-2ef76bc261be None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c7122477-1aae-4db2-8e35-2ef76bc261be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:12.231292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:12.232488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:12.233535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:12.233535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:12.233535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:12.233535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:12.233535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 285/560] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:12.234356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f176eb-fe4b-4607-920b-a34588973830 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.234643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f176eb-fe4b-4607-920b-a34588973830 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.234920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f176eb-fe4b-4607-920b-a34588973830 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.236361 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 286/561] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:12.236886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f176eb-fe4b-4607-920b-a34588973830 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:12.306346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90153960-17e4-48b7-9003-6eeaac365382 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:12.348990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242e953-1ded-4fcc-b404-009ef461d75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:12.350258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242e953-1ded-4fcc-b404-009ef461d75c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.350757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242e953-1ded-4fcc-b404-009ef461d75c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.351052 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242e953-1ded-4fcc-b404-009ef461d75c None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.351908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242e953-1ded-4fcc-b404-009ef461d75c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:12.352217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242e953-1ded-4fcc-b404-009ef461d75c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:12.353682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242e953-1ded-4fcc-b404-009ef461d75c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:12.354568 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f176eb-fe4b-4607-920b-a34588973830 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:12.375361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 276/562] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 4 09:02:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:12.376220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242e953-1ded-4fcc-b404-009ef461d75c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:12.403243 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e8d18-af8a-41a9-8694-603cff523c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:12.404061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e8d18-af8a-41a9-8694-603cff523c0e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.404350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e8d18-af8a-41a9-8694-603cff523c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.404629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e8d18-af8a-41a9-8694-603cff523c0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.407148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:12.416830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e8d18-af8a-41a9-8694-603cff523c0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:12.426763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cf9e8d18-af8a-41a9-8694-603cff523c0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:12.427991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:12.429110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:12.430043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 287/563] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:12.431304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362cd96-c9a5-4d1d-9bd3-08b38b165dd1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.431604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362cd96-c9a5-4d1d-9bd3-08b38b165dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.431938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362cd96-c9a5-4d1d-9bd3-08b38b165dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.451428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e362cd96-c9a5-4d1d-9bd3-08b38b165dd1 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:12.452970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e362cd96-c9a5-4d1d-9bd3-08b38b165dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:12.509671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242e953-1ded-4fcc-b404-009ef461d75c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:12.529497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 288/564] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:12.530941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee88e3-f7cb-4728-bc6b-1f9b30098235 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.531249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee88e3-f7cb-4728-bc6b-1f9b30098235 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.531509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee88e3-f7cb-4728-bc6b-1f9b30098235 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.532788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 289/565] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:12.533487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee88e3-f7cb-4728-bc6b-1f9b30098235 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:12.565464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b331b48-ef8f-4b76-aa1e-b82a15bcbc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:12.566248 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b331b48-ef8f-4b76-aa1e-b82a15bcbc06 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.566847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b331b48-ef8f-4b76-aa1e-b82a15bcbc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.567191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b331b48-ef8f-4b76-aa1e-b82a15bcbc06 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.568009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b331b48-ef8f-4b76-aa1e-b82a15bcbc06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:12.568320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b331b48-ef8f-4b76-aa1e-b82a15bcbc06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:12.569734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b331b48-ef8f-4b76-aa1e-b82a15bcbc06 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:12.582177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 277/566] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:12.582973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b331b48-ef8f-4b76-aa1e-b82a15bcbc06 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:12.650903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee88e3-f7cb-4728-bc6b-1f9b30098235 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:12.679968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:12.691808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcf6c1-d726-4567-91ec-47cc281a4d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:12.692648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf6c1-d726-4567-91ec-47cc281a4d25 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.692928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf6c1-d726-4567-91ec-47cc281a4d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.693202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf6c1-d726-4567-91ec-47cc281a4d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.698043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:12.701041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b331b48-ef8f-4b76-aa1e-b82a15bcbc06 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:12.704934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcf6c1-d726-4567-91ec-47cc281a4d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:12.706481 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcf6c1-d726-4567-91ec-47cc281a4d25 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-74fcf6c1-d726-4567-91ec-47cc281a4d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:12.725450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:12.726454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:12.727713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:12.727713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:12.727713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:12.727713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:12.727713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 290/567] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:12.728180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdd01c-65b9-4e82-8321-ede1e0cefb40 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.728180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdd01c-65b9-4e82-8321-ede1e0cefb40 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.728426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdd01c-65b9-4e82-8321-ede1e0cefb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.741390 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58ebce-0e14-4a6c-8427-e226924e5e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:12.742105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58ebce-0e14-4a6c-8427-e226924e5e23 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.742392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58ebce-0e14-4a6c-8427-e226924e5e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.742672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58ebce-0e14-4a6c-8427-e226924e5e23 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.743438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58ebce-0e14-4a6c-8427-e226924e5e23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:12.743750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58ebce-0e14-4a6c-8427-e226924e5e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:12.744752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-54fdd01c-65b9-4e82-8321-ede1e0cefb40 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:12.745124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58ebce-0e14-4a6c-8427-e226924e5e23 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:12.745979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-54fdd01c-65b9-4e82-8321-ede1e0cefb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:12.753747 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 278/568] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:12.754636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58ebce-0e14-4a6c-8427-e226924e5e23 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:12.812196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 291/569] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:12.812958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fdd01c-65b9-4e82-8321-ede1e0cefb40 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:12.858282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58ebce-0e14-4a6c-8427-e226924e5e23 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:12.896520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82425ae-c348-47e9-8dfe-b072b8a96d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:12.897327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82425ae-c348-47e9-8dfe-b072b8a96d4a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.897609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82425ae-c348-47e9-8dfe-b072b8a96d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.897913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82425ae-c348-47e9-8dfe-b072b8a96d4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.908511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82425ae-c348-47e9-8dfe-b072b8a96d4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c82425ae-c348-47e9-8dfe-b072b8a96d4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:12.916353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:12.917479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 279/570] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:12.918415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82425ae-c348-47e9-8dfe-b072b8a96d4a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:12.926512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fdd01c-65b9-4e82-8321-ede1e0cefb40 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:12.934183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:12.946758 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:12.962503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7ebd6-b536-4810-bec8-59ac11542704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:12.963305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7ebd6-b536-4810-bec8-59ac11542704 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:12.963586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7ebd6-b536-4810-bec8-59ac11542704 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:12.963865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7ebd6-b536-4810-bec8-59ac11542704 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:12.970470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7ebd6-b536-4810-bec8-59ac11542704 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:12.971908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7ebd6-b536-4810-bec8-59ac11542704 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e0b7ebd6-b536-4810-bec8-59ac11542704 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:12.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:12.986746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:12.987626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:12.987626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:12.987626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:12.987626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:12.987626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:12.987626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:12.987626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:12.987626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:12.987626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 292/571] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:12.988573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7ebd6-b536-4810-bec8-59ac11542704 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.029155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82425ae-c348-47e9-8dfe-b072b8a96d4a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.062087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdec1a5-3e9a-4d63-b0d1-1a10bc0a0ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.062807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdec1a5-3e9a-4d63-b0d1-1a10bc0a0ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.063026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdec1a5-3e9a-4d63-b0d1-1a10bc0a0ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.063296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdec1a5-3e9a-4d63-b0d1-1a10bc0a0ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.064019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdec1a5-3e9a-4d63-b0d1-1a10bc0a0ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:13.064321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdec1a5-3e9a-4d63-b0d1-1a10bc0a0ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:13.065704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdec1a5-3e9a-4d63-b0d1-1a10bc0a0ed4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:13.077452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 280/572] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:13.078098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdec1a5-3e9a-4d63-b0d1-1a10bc0a0ed4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.092779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7ebd6-b536-4810-bec8-59ac11542704 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.126670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e3990-5912-44e3-bdb6-3ba81259597e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.127822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e3990-5912-44e3-bdb6-3ba81259597e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.128102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e3990-5912-44e3-bdb6-3ba81259597e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.128376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e3990-5912-44e3-bdb6-3ba81259597e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.140171 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e3990-5912-44e3-bdb6-3ba81259597e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:13.141647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e3990-5912-44e3-bdb6-3ba81259597e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:13.157055 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ca7e3990-5912-44e3-bdb6-3ba81259597e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:13.159479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:13.160668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:13.161826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:13.161826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:13.161826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:13.161826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:13.165420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 293/573] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:13.166271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e3990-5912-44e3-bdb6-3ba81259597e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.177639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdec1a5-3e9a-4d63-b0d1-1a10bc0a0ed4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.186499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:13.212721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aadd6a-3731-493e-ad46-c0e36c473c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.213524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aadd6a-3731-493e-ad46-c0e36c473c64 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.213808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aadd6a-3731-493e-ad46-c0e36c473c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.214116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aadd6a-3731-493e-ad46-c0e36c473c64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.224557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aadd6a-3731-493e-ad46-c0e36c473c64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-82aadd6a-3731-493e-ad46-c0e36c473c64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:13.232234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:13.233170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:13.234059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 281/574] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:13.234770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aadd6a-3731-493e-ad46-c0e36c473c64 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.274179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e3990-5912-44e3-bdb6-3ba81259597e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.307181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb461259-4a63-4cd2-8aa4-db9a8610903b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.307852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461259-4a63-4cd2-8aa4-db9a8610903b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.308143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461259-4a63-4cd2-8aa4-db9a8610903b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.308426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461259-4a63-4cd2-8aa4-db9a8610903b None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.309160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb461259-4a63-4cd2-8aa4-db9a8610903b None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:13.309472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb461259-4a63-4cd2-8aa4-db9a8610903b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:13.310898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb461259-4a63-4cd2-8aa4-db9a8610903b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:13.319030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 294/575] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:13.322725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb461259-4a63-4cd2-8aa4-db9a8610903b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.339951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aadd6a-3731-493e-ad46-c0e36c473c64 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.372989 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b799cb5-27f2-431c-aecc-ea290f82a314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.373657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b799cb5-27f2-431c-aecc-ea290f82a314 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.373948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b799cb5-27f2-431c-aecc-ea290f82a314 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.374232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b799cb5-27f2-431c-aecc-ea290f82a314 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.375024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b799cb5-27f2-431c-aecc-ea290f82a314 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:13.375334 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b799cb5-27f2-431c-aecc-ea290f82a314 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:13.376778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b799cb5-27f2-431c-aecc-ea290f82a314 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:13.389150 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 282/576] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:13.389747 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b799cb5-27f2-431c-aecc-ea290f82a314 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.392252 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:13.405783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:13.425437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb461259-4a63-4cd2-8aa4-db9a8610903b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.459526 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b95f7-99c5-49c1-897f-9ce9d8d757c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.460332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b95f7-99c5-49c1-897f-9ce9d8d757c8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.461293 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b95f7-99c5-49c1-897f-9ce9d8d757c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.461293 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b95f7-99c5-49c1-897f-9ce9d8d757c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.467661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b95f7-99c5-49c1-897f-9ce9d8d757c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:13.469101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b95f7-99c5-49c1-897f-9ce9d8d757c8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cb8b95f7-99c5-49c1-897f-9ce9d8d757c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:13.483469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:13.484399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:13.485304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:13.485304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:13.485304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:13.485304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:13.485304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:13.485304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:13.485304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:13.485304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:13.485304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 295/577] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:13.485823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b95f7-99c5-49c1-897f-9ce9d8d757c8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.491549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b799cb5-27f2-431c-aecc-ea290f82a314 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.525775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d970d-432f-4e97-8c4e-9c045236bfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.526463 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d970d-432f-4e97-8c4e-9c045236bfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.526744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d970d-432f-4e97-8c4e-9c045236bfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.527015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d970d-432f-4e97-8c4e-9c045236bfb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.527735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d970d-432f-4e97-8c4e-9c045236bfb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:13.528434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d970d-432f-4e97-8c4e-9c045236bfb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:13.529808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d970d-432f-4e97-8c4e-9c045236bfb3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:13.541674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 283/578] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:13.542644 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d970d-432f-4e97-8c4e-9c045236bfb3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.597312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b95f7-99c5-49c1-897f-9ce9d8d757c8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.607086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:13.631352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851c2be-3cd7-476a-af1b-f0e5a3cc6b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.632054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851c2be-3cd7-476a-af1b-f0e5a3cc6b2c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.632335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851c2be-3cd7-476a-af1b-f0e5a3cc6b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.632609 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851c2be-3cd7-476a-af1b-f0e5a3cc6b2c None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.633359 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851c2be-3cd7-476a-af1b-f0e5a3cc6b2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:13.633669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851c2be-3cd7-476a-af1b-f0e5a3cc6b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:13.635074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851c2be-3cd7-476a-af1b-f0e5a3cc6b2c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:13.643244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 296/579] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:13.644085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851c2be-3cd7-476a-af1b-f0e5a3cc6b2c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.646272 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d970d-432f-4e97-8c4e-9c045236bfb3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.647071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:13.684211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428fd7d-bc60-40e9-b898-df5448157110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.685140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428fd7d-bc60-40e9-b898-df5448157110 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.685421 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428fd7d-bc60-40e9-b898-df5448157110 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.685694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428fd7d-bc60-40e9-b898-df5448157110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.695533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428fd7d-bc60-40e9-b898-df5448157110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:13.696990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428fd7d-bc60-40e9-b898-df5448157110 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c428fd7d-bc60-40e9-b898-df5448157110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:13.714070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:13.715147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:13.716385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:13.716385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:13.716385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:13.716385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:13.716385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 284/580] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:13.716385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428fd7d-bc60-40e9-b898-df5448157110 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.749100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851c2be-3cd7-476a-af1b-f0e5a3cc6b2c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.782945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1823a1f-4e6c-4a64-b5b7-ada86fc89497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.783589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1823a1f-4e6c-4a64-b5b7-ada86fc89497 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.783867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1823a1f-4e6c-4a64-b5b7-ada86fc89497 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.784141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1823a1f-4e6c-4a64-b5b7-ada86fc89497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.784996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1823a1f-4e6c-4a64-b5b7-ada86fc89497 None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=8cbb65dbfadf4c63a272f5318efa875a) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:13.785311 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1823a1f-4e6c-4a64-b5b7-ada86fc89497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:13.786799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1823a1f-4e6c-4a64-b5b7-ada86fc89497 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:13.792150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 297/581] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/role_assignments?user.id=8cbb65dbfadf4c63a272f5318efa875a&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:13.792910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1823a1f-4e6c-4a64-b5b7-ada86fc89497 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.834039 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428fd7d-bc60-40e9-b898-df5448157110 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.851962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:13.867488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:13.875048 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7094-a20b-49e5-8183-88f207dfa385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.875834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7094-a20b-49e5-8183-88f207dfa385 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.876189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7094-a20b-49e5-8183-88f207dfa385 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.876534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7094-a20b-49e5-8183-88f207dfa385 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.877772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f7094-a20b-49e5-8183-88f207dfa385 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:13.878180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f7094-a20b-49e5-8183-88f207dfa385 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:13.879827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f7094-a20b-49e5-8183-88f207dfa385 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:13.898523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 285/582] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:13.898523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7094-a20b-49e5-8183-88f207dfa385 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:13.914035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1823a1f-4e6c-4a64-b5b7-ada86fc89497 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:13.952401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1132bfe-2879-4e54-ba13-656b779b44f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:13.953318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132bfe-2879-4e54-ba13-656b779b44f9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:13.953661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132bfe-2879-4e54-ba13-656b779b44f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:13.954017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1132bfe-2879-4e54-ba13-656b779b44f9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:13.962889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1132bfe-2879-4e54-ba13-656b779b44f9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:13.964497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1132bfe-2879-4e54-ba13-656b779b44f9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d1132bfe-2879-4e54-ba13-656b779b44f9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:13.979228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:13.980415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:13.981584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:13.981584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:13.981584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:13.981584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:13.981584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:13.981584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:13.981584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:13.981584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:13.981584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:02:13.981584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:13.982463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 298/583] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:13.983247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1132bfe-2879-4e54-ba13-656b779b44f9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:14.009017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7094-a20b-49e5-8183-88f207dfa385 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:14.046114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6efc40-3971-4adc-a671-e1f4f108453f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:14.047009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6efc40-3971-4adc-a671-e1f4f108453f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:14.047358 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6efc40-3971-4adc-a671-e1f4f108453f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:14.047824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6efc40-3971-4adc-a671-e1f4f108453f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:14.058522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6efc40-3971-4adc-a671-e1f4f108453f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:14.060062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6efc40-3971-4adc-a671-e1f4f108453f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-dc6efc40-3971-4adc-a671-e1f4f108453f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:14.077791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:14.078936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:14.080042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:14.080042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:14.080042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:14.080042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:14.080626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 286/584] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:14.081390 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6efc40-3971-4adc-a671-e1f4f108453f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:14.095984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1132bfe-2879-4e54-ba13-656b779b44f9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:14.112508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:14.124254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:14.137057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4239f0-bcfa-495e-a376-df90170e6c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:14.137746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4239f0-bcfa-495e-a376-df90170e6c09 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:14.138117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4239f0-bcfa-495e-a376-df90170e6c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:14.138345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4239f0-bcfa-495e-a376-df90170e6c09 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:14.139100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4239f0-bcfa-495e-a376-df90170e6c09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:14.139400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4239f0-bcfa-495e-a376-df90170e6c09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:14.140788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4239f0-bcfa-495e-a376-df90170e6c09 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:14.153671 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 299/585] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:14.154744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4239f0-bcfa-495e-a376-df90170e6c09 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:14.201689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6efc40-3971-4adc-a671-e1f4f108453f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:14.244460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafe2b5-1cd2-4c96-8ee6-db4a4e5317e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:14.245370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafe2b5-1cd2-4c96-8ee6-db4a4e5317e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:14.245658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafe2b5-1cd2-4c96-8ee6-db4a4e5317e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:14.245966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafe2b5-1cd2-4c96-8ee6-db4a4e5317e2 None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/739f3b5eaef542c29a0cf0b475c0d965/roles/f0e6d5b558954227a2e1423c3d1e5d18` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:14.262074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafe2b5-1cd2-4c96-8ee6-db4a4e5317e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=739f3b5eaef542c29a0cf0b475c0d965, role_id=f0e6d5b558954227a2e1423c3d1e5d18)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:14.263394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4239f0-bcfa-495e-a376-df90170e6c09 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:14.263567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafe2b5-1cd2-4c96-8ee6-db4a4e5317e2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:14.285916 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 287/586] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:02:14 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/739f3b5eaef542c29a0cf0b475c0d965/roles/f0e6d5b558954227a2e1423c3d1e5d18 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:02:14.285916 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafe2b5-1cd2-4c96-8ee6-db4a4e5317e2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:14.316089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eadd67-1f8d-41d2-8195-d3a138265c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:14.317080 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eadd67-1f8d-41d2-8195-d3a138265c5e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:14.317367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eadd67-1f8d-41d2-8195-d3a138265c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:14.317974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eadd67-1f8d-41d2-8195-d3a138265c5e None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:14.320657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eadd67-1f8d-41d2-8195-d3a138265c5e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:14.321000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eadd67-1f8d-41d2-8195-d3a138265c5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:14.322618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eadd67-1f8d-41d2-8195-d3a138265c5e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:14.345783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 300/587] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 4 09:02:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:14.346786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eadd67-1f8d-41d2-8195-d3a138265c5e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:14.370553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:14.414049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:14.416003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafe2b5-1cd2-4c96-8ee6-db4a4e5317e2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:14.467615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758513a9-352a-4b2a-ac45-2e80f0c25e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:14.468294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758513a9-352a-4b2a-ac45-2e80f0c25e0a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:14.468575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758513a9-352a-4b2a-ac45-2e80f0c25e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:14.468848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758513a9-352a-4b2a-ac45-2e80f0c25e0a None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:14.470314 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758513a9-352a-4b2a-ac45-2e80f0c25e0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:14.470864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758513a9-352a-4b2a-ac45-2e80f0c25e0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:14.472652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758513a9-352a-4b2a-ac45-2e80f0c25e0a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:14.482757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eadd67-1f8d-41d2-8195-d3a138265c5e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:14.486387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 288/588] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:14.487500 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758513a9-352a-4b2a-ac45-2e80f0c25e0a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:14.526333 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21442057-32a5-4c5c-8840-44f50ff9f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:14.527146 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21442057-32a5-4c5c-8840-44f50ff9f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:14.527427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21442057-32a5-4c5c-8840-44f50ff9f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:14.527700 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21442057-32a5-4c5c-8840-44f50ff9f8f0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:14.535930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21442057-32a5-4c5c-8840-44f50ff9f8f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:14.537460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21442057-32a5-4c5c-8840-44f50ff9f8f0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-21442057-32a5-4c5c-8840-44f50ff9f8f0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:14.553670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:14.554694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:14.556008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:14.556008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:14.556008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:14.556008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:14.556008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:14.556008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:14.556008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:14.556008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:14.556008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:02:14.556008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:14.556008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 301/589] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:14.556739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21442057-32a5-4c5c-8840-44f50ff9f8f0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:14.611404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758513a9-352a-4b2a-ac45-2e80f0c25e0a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:14.658284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410ed83-9411-47cf-a8be-5e20f6be04c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:14.659666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410ed83-9411-47cf-a8be-5e20f6be04c5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:14.660189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410ed83-9411-47cf-a8be-5e20f6be04c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:14.660623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410ed83-9411-47cf-a8be-5e20f6be04c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:14.664485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:14.677918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410ed83-9411-47cf-a8be-5e20f6be04c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:14.679418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410ed83-9411-47cf-a8be-5e20f6be04c5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:14.681682 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21442057-32a5-4c5c-8840-44f50ff9f8f0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:14.686210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c410ed83-9411-47cf-a8be-5e20f6be04c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:14.708628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:14.709682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:14.710630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:14.710630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:14.710630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:14.710630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:14.711611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 289/590] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:14.712484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410ed83-9411-47cf-a8be-5e20f6be04c5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:14.726743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd4856-b6cd-41a5-9397-1d466e07bffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:14.728309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd4856-b6cd-41a5-9397-1d466e07bffa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:14.728857 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd4856-b6cd-41a5-9397-1d466e07bffa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:14.729625 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd4856-b6cd-41a5-9397-1d466e07bffa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:14.742388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd4856-b6cd-41a5-9397-1d466e07bffa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:14.745435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd4856-b6cd-41a5-9397-1d466e07bffa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9bdd4856-b6cd-41a5-9397-1d466e07bffa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:14.768785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:14.770010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:14.770945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:14.770945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:14.770945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:14.770945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:14.770945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:14.770945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:14.770945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:14.770945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:14.770945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:02:14.770945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:14.773903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 302/591] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:14.775194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd4856-b6cd-41a5-9397-1d466e07bffa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:14.830584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410ed83-9411-47cf-a8be-5e20f6be04c5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:14.874936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae2dd4-df59-4677-bffe-6823193df9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:14.876238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2dd4-df59-4677-bffe-6823193df9ec None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:14.876667 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2dd4-df59-4677-bffe-6823193df9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:14.877464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae2dd4-df59-4677-bffe-6823193df9ec None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:14.878495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae2dd4-df59-4677-bffe-6823193df9ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:14.879011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae2dd4-df59-4677-bffe-6823193df9ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:14.881419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae2dd4-df59-4677-bffe-6823193df9ec None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:14.896254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd4856-b6cd-41a5-9397-1d466e07bffa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:14.911020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 290/592] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 4 09:02:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:14.911020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae2dd4-df59-4677-bffe-6823193df9ec None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:14.942417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f1371-1b8c-4416-96f0-432ab09c5958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:14.943087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f1371-1b8c-4416-96f0-432ab09c5958 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:14.943371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f1371-1b8c-4416-96f0-432ab09c5958 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:14.943653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f1371-1b8c-4416-96f0-432ab09c5958 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:14.944385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f1371-1b8c-4416-96f0-432ab09c5958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:14.944696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f1371-1b8c-4416-96f0-432ab09c5958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:14.947008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f1371-1b8c-4416-96f0-432ab09c5958 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:14.965402 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 303/593] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:14.967744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f1371-1b8c-4416-96f0-432ab09c5958 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:14.980493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:15.000823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:15.055580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae2dd4-df59-4677-bffe-6823193df9ec None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:15.089241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f1371-1b8c-4416-96f0-432ab09c5958 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:15.095347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca1264-be5f-4ec1-a278-6a922c068744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:15.096032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca1264-be5f-4ec1-a278-6a922c068744 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:15.096318 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca1264-be5f-4ec1-a278-6a922c068744 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:15.096599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca1264-be5f-4ec1-a278-6a922c068744 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:15.097525 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca1264-be5f-4ec1-a278-6a922c068744 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:15.098473 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca1264-be5f-4ec1-a278-6a922c068744 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:15.099930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca1264-be5f-4ec1-a278-6a922c068744 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:15.118801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 291/594] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jul 4 09:02:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:15.119691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca1264-be5f-4ec1-a278-6a922c068744 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:15.139109 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fc2ce-a2d1-40d6-b49e-94d260172a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:15.140654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fc2ce-a2d1-40d6-b49e-94d260172a8b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:15.141166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fc2ce-a2d1-40d6-b49e-94d260172a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:15.141643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fc2ce-a2d1-40d6-b49e-94d260172a8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:15.157983 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fc2ce-a2d1-40d6-b49e-94d260172a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:15.160858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fc2ce-a2d1-40d6-b49e-94d260172a8b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-736fc2ce-a2d1-40d6-b49e-94d260172a8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:15.183427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:15.184502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:15.185634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:15.185634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:15.185634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:15.185634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:15.192665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 304/595] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:15.192665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fc2ce-a2d1-40d6-b49e-94d260172a8b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:15.245498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca1264-be5f-4ec1-a278-6a922c068744 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:15.248813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:15.286131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea495c50-fa31-4385-9f05-665104debf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:15.286976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea495c50-fa31-4385-9f05-665104debf32 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:15.287273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea495c50-fa31-4385-9f05-665104debf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:15.287552 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea495c50-fa31-4385-9f05-665104debf32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:15.287860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:15.295437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea495c50-fa31-4385-9f05-665104debf32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:15.296844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea495c50-fa31-4385-9f05-665104debf32 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ea495c50-fa31-4385-9f05-665104debf32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:15.313634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:15.314613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:15.316126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:15.316126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:15.316126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:15.316126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:15.316126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:15.316126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:15.316126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:15.316126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:15.316126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 292/596] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:15.316126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea495c50-fa31-4385-9f05-665104debf32 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:15.318270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fc2ce-a2d1-40d6-b49e-94d260172a8b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:15.366817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83055192-904d-429c-95a1-a21d9f5defd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:15.368593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83055192-904d-429c-95a1-a21d9f5defd3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:15.369091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83055192-904d-429c-95a1-a21d9f5defd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:15.370019 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83055192-904d-429c-95a1-a21d9f5defd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:15.384871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83055192-904d-429c-95a1-a21d9f5defd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:15.387642 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83055192-904d-429c-95a1-a21d9f5defd3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-83055192-904d-429c-95a1-a21d9f5defd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:15.409140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:15.410194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:15.411090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:15.411090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:15.411090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:15.411090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:15.411880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 305/597] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:15.412394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83055192-904d-429c-95a1-a21d9f5defd3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:15.441710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea495c50-fa31-4385-9f05-665104debf32 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:15.490042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c28f0-7ab6-4328-8a98-8410fcdaf8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:15.491006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c28f0-7ab6-4328-8a98-8410fcdaf8ee None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:15.491465 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c28f0-7ab6-4328-8a98-8410fcdaf8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:15.491887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c28f0-7ab6-4328-8a98-8410fcdaf8ee None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:15.492781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c28f0-7ab6-4328-8a98-8410fcdaf8ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:15.493233 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c28f0-7ab6-4328-8a98-8410fcdaf8ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:15.495235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c28f0-7ab6-4328-8a98-8410fcdaf8ee None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:15.513317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 293/598] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:15.513317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c28f0-7ab6-4328-8a98-8410fcdaf8ee None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:15.533641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83055192-904d-429c-95a1-a21d9f5defd3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:15.540464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:15.571475 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:15.583146 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf902f4-b654-4b54-b27a-925236f710f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:15.583817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf902f4-b654-4b54-b27a-925236f710f8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:15.584094 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf902f4-b654-4b54-b27a-925236f710f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:15.584364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf902f4-b654-4b54-b27a-925236f710f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:15.585153 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf902f4-b654-4b54-b27a-925236f710f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:15.585463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf902f4-b654-4b54-b27a-925236f710f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:15.586947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf902f4-b654-4b54-b27a-925236f710f8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:15.604991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 306/599] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:15.605814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf902f4-b654-4b54-b27a-925236f710f8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:15.634695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c28f0-7ab6-4328-8a98-8410fcdaf8ee None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:15.679356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e4734-3898-4c0c-9946-3282bfc41635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:15.680451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4734-3898-4c0c-9946-3282bfc41635 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:15.680933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4734-3898-4c0c-9946-3282bfc41635 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:15.681402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4734-3898-4c0c-9946-3282bfc41635 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:15.682626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e4734-3898-4c0c-9946-3282bfc41635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:15.682944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e4734-3898-4c0c-9946-3282bfc41635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:15.684381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e4734-3898-4c0c-9946-3282bfc41635 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:15.702126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 294/600] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:15.702894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e4734-3898-4c0c-9946-3282bfc41635 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:15.712172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf902f4-b654-4b54-b27a-925236f710f8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:15.761979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcd90e-b56c-4037-ba45-b4456b883fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:15.763407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcd90e-b56c-4037-ba45-b4456b883fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:15.764304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcd90e-b56c-4037-ba45-b4456b883fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:15.765031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcd90e-b56c-4037-ba45-b4456b883fd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:15.774977 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcd90e-b56c-4037-ba45-b4456b883fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:15.777380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcd90e-b56c-4037-ba45-b4456b883fd6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:15.793190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d3dcd90e-b56c-4037-ba45-b4456b883fd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:15.796301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:15.797294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:15.798231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:15.798231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:15.798231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:15.798231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:15.798231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:15.798231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:15.798231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:15.798231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:15.804365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 307/601] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:15.805246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcd90e-b56c-4037-ba45-b4456b883fd6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:15.814322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e4734-3898-4c0c-9946-3282bfc41635 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:15.850968 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5e0b1-11a2-4b4f-9eef-ba10d5b3dd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:15.852198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5e0b1-11a2-4b4f-9eef-ba10d5b3dd0a None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:15.852773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5e0b1-11a2-4b4f-9eef-ba10d5b3dd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:15.853074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5e0b1-11a2-4b4f-9eef-ba10d5b3dd0a None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/e9f34c35a9ba4376a9b55ec5e23cf35f/roles/f0e6d5b558954227a2e1423c3d1e5d18` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:15.859453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:15.868599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5e0b1-11a2-4b4f-9eef-ba10d5b3dd0a None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=e9f34c35a9ba4376a9b55ec5e23cf35f, role_id=f0e6d5b558954227a2e1423c3d1e5d18)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:15.869943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5e0b1-11a2-4b4f-9eef-ba10d5b3dd0a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:15.884657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 295/602] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:02:15 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/e9f34c35a9ba4376a9b55ec5e23cf35f/roles/f0e6d5b558954227a2e1423c3d1e5d18 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:02:15.885477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5e0b1-11a2-4b4f-9eef-ba10d5b3dd0a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:15.920581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcd90e-b56c-4037-ba45-b4456b883fd6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:15.966927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfac310-38aa-4104-95b3-e5e8552b3e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:15.969282 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfac310-38aa-4104-95b3-e5e8552b3e5c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:15.969851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfac310-38aa-4104-95b3-e5e8552b3e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:15.970641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfac310-38aa-4104-95b3-e5e8552b3e5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:15.981877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfac310-38aa-4104-95b3-e5e8552b3e5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:15.984535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfac310-38aa-4104-95b3-e5e8552b3e5c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8cfac310-38aa-4104-95b3-e5e8552b3e5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:16.009119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:16.010174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:16.011114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:16.011114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:16.011114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:16.011114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:16.011114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:16.011114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:16.011114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:16.011114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:16.012530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 308/603] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:16.014283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5e0b1-11a2-4b4f-9eef-ba10d5b3dd0a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:16.014714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af315cf-b1e7-4976-aef7-fe8ff7c923b7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:16.015210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af315cf-b1e7-4976-aef7-fe8ff7c923b7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:16.015788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af315cf-b1e7-4976-aef7-fe8ff7c923b7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:16.018991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 309/604] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:16.023618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af315cf-b1e7-4976-aef7-fe8ff7c923b7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:16.068247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b7f6c-5081-4b1a-9736-84f8725b61b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:16.068927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b7f6c-5081-4b1a-9736-84f8725b61b5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:16.069263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b7f6c-5081-4b1a-9736-84f8725b61b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:16.069980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b7f6c-5081-4b1a-9736-84f8725b61b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:16.070797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b7f6c-5081-4b1a-9736-84f8725b61b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:16.071112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b7f6c-5081-4b1a-9736-84f8725b61b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:16.072958 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b7f6c-5081-4b1a-9736-84f8725b61b5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:16.075410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:16.093893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 296/605] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:16.093893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca242a-5c5a-4b3a-a60c-2f908f5fdfae None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:16.093893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca242a-5c5a-4b3a-a60c-2f908f5fdfae None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:16.093893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca242a-5c5a-4b3a-a60c-2f908f5fdfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:16.114272 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-30ca242a-5c5a-4b3a-a60c-2f908f5fdfae None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:16.115599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-30ca242a-5c5a-4b3a-a60c-2f908f5fdfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:16.141770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af315cf-b1e7-4976-aef7-fe8ff7c923b7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:16.152784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:16.181404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fbd90-f960-4c8c-9753-36a5db486d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:16.182467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fbd90-f960-4c8c-9753-36a5db486d64 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:16.182969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fbd90-f960-4c8c-9753-36a5db486d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:16.183576 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fbd90-f960-4c8c-9753-36a5db486d64 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:16.184604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fbd90-f960-4c8c-9753-36a5db486d64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:16.185102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fbd90-f960-4c8c-9753-36a5db486d64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:16.186907 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fbd90-f960-4c8c-9753-36a5db486d64 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:16.199836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 310/606] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:16.205447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fbd90-f960-4c8c-9753-36a5db486d64 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:16.208306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 297/607] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:16.210265 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca242a-5c5a-4b3a-a60c-2f908f5fdfae None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:16.331512 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca242a-5c5a-4b3a-a60c-2f908f5fdfae None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:16.347614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fbd90-f960-4c8c-9753-36a5db486d64 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:16.358829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:16.373259 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d967f64-404f-48bf-b60d-31087a29d1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:16.374617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d967f64-404f-48bf-b60d-31087a29d1b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:16.375110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d967f64-404f-48bf-b60d-31087a29d1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:16.375620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d967f64-404f-48bf-b60d-31087a29d1b6 None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/98649aa38f764af0b41ff5594b94a989/roles/f0e6d5b558954227a2e1423c3d1e5d18` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:16.388884 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61080cc3-9bda-4f94-9202-362aa2f2e7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:16.389964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61080cc3-9bda-4f94-9202-362aa2f2e7a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:16.390259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61080cc3-9bda-4f94-9202-362aa2f2e7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:16.390537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61080cc3-9bda-4f94-9202-362aa2f2e7a6 None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/107a90e384504e8d80264d6bf20f1d9b/roles/f0e6d5b558954227a2e1423c3d1e5d18` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:16.394912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d967f64-404f-48bf-b60d-31087a29d1b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=98649aa38f764af0b41ff5594b94a989, role_id=f0e6d5b558954227a2e1423c3d1e5d18)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:16.396668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d967f64-404f-48bf-b60d-31087a29d1b6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:16.408615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61080cc3-9bda-4f94-9202-362aa2f2e7a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=107a90e384504e8d80264d6bf20f1d9b, role_id=f0e6d5b558954227a2e1423c3d1e5d18)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:16.410493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61080cc3-9bda-4f94-9202-362aa2f2e7a6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:16.421920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 298/608] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:02:16 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/98649aa38f764af0b41ff5594b94a989/roles/f0e6d5b558954227a2e1423c3d1e5d18 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:02:16.421920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d967f64-404f-48bf-b60d-31087a29d1b6 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:16.436888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 311/609] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:02:16 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/107a90e384504e8d80264d6bf20f1d9b/roles/f0e6d5b558954227a2e1423c3d1e5d18 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:02:16.477836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8015c-4ca9-4d72-b14c-cd3f7bd807e7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:16.478209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8015c-4ca9-4d72-b14c-cd3f7bd807e7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:16.478425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8015c-4ca9-4d72-b14c-cd3f7bd807e7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:16.480069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 312/610] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:16.486166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20d182-272c-4962-b4f0-c5acdb39d69c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:16.486528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20d182-272c-4962-b4f0-c5acdb39d69c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:16.486785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20d182-272c-4962-b4f0-c5acdb39d69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:16.515474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4a20d182-272c-4962-b4f0-c5acdb39d69c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:16.516733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4a20d182-272c-4962-b4f0-c5acdb39d69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:16.557043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:16.621001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d967f64-404f-48bf-b60d-31087a29d1b6 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:16.667001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b6eb5-69fb-4d28-894f-0f299b29255d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:16.668364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b6eb5-69fb-4d28-894f-0f299b29255d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:16.668906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b6eb5-69fb-4d28-894f-0f299b29255d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:16.669185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b6eb5-69fb-4d28-894f-0f299b29255d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:16.678034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 313/611] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:02:16.683333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b6eb5-69fb-4d28-894f-0f299b29255d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-8b4b6eb5-69fb-4d28-894f-0f299b29255d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:16.691323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:16.692685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:16.693549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 299/612] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:16.699105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20d182-272c-4962-b4f0-c5acdb39d69c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:16.711167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b6eb5-69fb-4d28-894f-0f299b29255d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:16.760000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:16.884859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b6eb5-69fb-4d28-894f-0f299b29255d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:16.891596 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20d182-272c-4962-b4f0-c5acdb39d69c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:16.959447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7658e-c1ff-4ca5-b74e-fe1db28ac97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:16.960300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7658e-c1ff-4ca5-b74e-fe1db28ac97d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:16.960581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7658e-c1ff-4ca5-b74e-fe1db28ac97d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:16.960861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7658e-c1ff-4ca5-b74e-fe1db28ac97d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:16.966766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ce095-5b39-4540-adf7-ada39a6846ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:16.967458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ce095-5b39-4540-adf7-ada39a6846ce None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:16.967739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ce095-5b39-4540-adf7-ada39a6846ce None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:16.968020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ce095-5b39-4540-adf7-ada39a6846ce None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:16.968788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ce095-5b39-4540-adf7-ada39a6846ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:16.969100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ce095-5b39-4540-adf7-ada39a6846ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:16.970549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ce095-5b39-4540-adf7-ada39a6846ce None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:16.982176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7658e-c1ff-4ca5-b74e-fe1db28ac97d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:16.997785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 314/613] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d7a7658e-c1ff-4ca5-b74e-fe1db28ac97d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:17.000056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:17.001001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:17.002643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 300/614] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:17.004761 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ce095-5b39-4540-adf7-ada39a6846ce None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:17.009023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7658e-c1ff-4ca5-b74e-fe1db28ac97d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:17.043089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:17.155462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:17.172864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7658e-c1ff-4ca5-b74e-fe1db28ac97d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:17.224422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ce095-5b39-4540-adf7-ada39a6846ce None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:17.242816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5023a-838b-4ee8-a97f-3a0a65f7893d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:17.243516 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5023a-838b-4ee8-a97f-3a0a65f7893d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:17.243799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5023a-838b-4ee8-a97f-3a0a65f7893d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:17.244074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5023a-838b-4ee8-a97f-3a0a65f7893d None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:17.244839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5023a-838b-4ee8-a97f-3a0a65f7893d None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:17.245145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5023a-838b-4ee8-a97f-3a0a65f7893d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:17.246567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5023a-838b-4ee8-a97f-3a0a65f7893d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:17.265334 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 301/615] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:17.271418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5023a-838b-4ee8-a97f-3a0a65f7893d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:17.294920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8437b-a61a-4fae-83d2-c950be9ba15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:17.295745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8437b-a61a-4fae-83d2-c950be9ba15b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:17.296029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8437b-a61a-4fae-83d2-c950be9ba15b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:17.296304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8437b-a61a-4fae-83d2-c950be9ba15b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:17.318415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8437b-a61a-4fae-83d2-c950be9ba15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:17.320251 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8437b-a61a-4fae-83d2-c950be9ba15b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f0f8437b-a61a-4fae-83d2-c950be9ba15b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:17.351043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:17.352463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:17.353370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:17.353370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:17.353370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:17.353370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:17.358357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 315/616] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:17.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8437b-a61a-4fae-83d2-c950be9ba15b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:17.415166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5023a-838b-4ee8-a97f-3a0a65f7893d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:17.465562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480eeb5-0d64-45fa-a220-a4fb024331b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:17.466391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480eeb5-0d64-45fa-a220-a4fb024331b5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:17.466669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480eeb5-0d64-45fa-a220-a4fb024331b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:17.466939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480eeb5-0d64-45fa-a220-a4fb024331b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:17.478509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:17.492155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480eeb5-0d64-45fa-a220-a4fb024331b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:17.493609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480eeb5-0d64-45fa-a220-a4fb024331b5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-7480eeb5-0d64-45fa-a220-a4fb024331b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:17.521189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:17.522187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:17.523121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:17.523121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:17.523121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:17.523121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:17.528077 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 302/617] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:17.530879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480eeb5-0d64-45fa-a220-a4fb024331b5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:17.541545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8437b-a61a-4fae-83d2-c950be9ba15b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:17.550979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:17.581198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d9d45-6144-4134-b8b5-561424aa8664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:17.582240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d9d45-6144-4134-b8b5-561424aa8664 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:17.582529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d9d45-6144-4134-b8b5-561424aa8664 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:17.582806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d9d45-6144-4134-b8b5-561424aa8664 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:17.583543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d9d45-6144-4134-b8b5-561424aa8664 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:17.583850 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d9d45-6144-4134-b8b5-561424aa8664 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:17.586021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d9d45-6144-4134-b8b5-561424aa8664 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:17.615241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 316/618] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:17.623008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d9d45-6144-4134-b8b5-561424aa8664 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:17.683339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480eeb5-0d64-45fa-a220-a4fb024331b5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:17.737541 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6719f4-3be5-470a-8ce0-b3ccbba1feeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:17.738414 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6719f4-3be5-470a-8ce0-b3ccbba1feeb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:17.738706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6719f4-3be5-470a-8ce0-b3ccbba1feeb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:17.738982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6719f4-3be5-470a-8ce0-b3ccbba1feeb None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:17.739734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6719f4-3be5-470a-8ce0-b3ccbba1feeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:17.740045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6719f4-3be5-470a-8ce0-b3ccbba1feeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:17.741466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6719f4-3be5-470a-8ce0-b3ccbba1feeb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:17.768001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 303/619] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:17.773679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6719f4-3be5-470a-8ce0-b3ccbba1feeb None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:17.782040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d9d45-6144-4134-b8b5-561424aa8664 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:17.826568 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:17.856817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c545ed-6b7f-4e93-9806-531a59628870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:17.857938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c545ed-6b7f-4e93-9806-531a59628870 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:17.858397 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c545ed-6b7f-4e93-9806-531a59628870 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:17.858817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c545ed-6b7f-4e93-9806-531a59628870 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:17.869757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c545ed-6b7f-4e93-9806-531a59628870 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:17.873294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c545ed-6b7f-4e93-9806-531a59628870 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:17.883818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b5c545ed-6b7f-4e93-9806-531a59628870 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:17.904662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:17.906088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:17.907303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:17.907303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:17.907303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:17.907303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:17.907303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:17.907303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:17.907303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:17.907303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:17.907303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:17.907303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:17.907985 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 317/620] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:17.914904 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c545ed-6b7f-4e93-9806-531a59628870 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:17.958854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6719f4-3be5-470a-8ce0-b3ccbba1feeb None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:18.005568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c83923-1c90-40df-ab65-adbc85f97276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:18.006678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c83923-1c90-40df-ab65-adbc85f97276 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.008635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c83923-1c90-40df-ab65-adbc85f97276 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.009051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c83923-1c90-40df-ab65-adbc85f97276 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.021639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c83923-1c90-40df-ab65-adbc85f97276 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:18.023391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c83923-1c90-40df-ab65-adbc85f97276 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-50c83923-1c90-40df-ab65-adbc85f97276 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:18.046506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:18.047678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:18.048799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:18.048799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:18.048799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:18.048799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:18.048799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:18.048799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:18.048799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:18.048799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:18.048799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:18.048799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:18.051091 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 304/621] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 4 09:02:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:18.060980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c83923-1c90-40df-ab65-adbc85f97276 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:18.101494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c545ed-6b7f-4e93-9806-531a59628870 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:18.172501 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beea58-7721-4885-9b95-eb22e0d840cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:18.173190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beea58-7721-4885-9b95-eb22e0d840cf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.173476 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beea58-7721-4885-9b95-eb22e0d840cf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.173751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beea58-7721-4885-9b95-eb22e0d840cf None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.174630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69beea58-7721-4885-9b95-eb22e0d840cf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:18.174939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69beea58-7721-4885-9b95-eb22e0d840cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:18.176616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69beea58-7721-4885-9b95-eb22e0d840cf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:18.204464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 318/622] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 4 09:02:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:18.210974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beea58-7721-4885-9b95-eb22e0d840cf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:18.251218 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c83923-1c90-40df-ab65-adbc85f97276 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:18.277414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:18.288154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:18.298164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039c20c-8712-4b83-82a7-34aed21fec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:18.299097 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039c20c-8712-4b83-82a7-34aed21fec88 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.299809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039c20c-8712-4b83-82a7-34aed21fec88 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.300164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039c20c-8712-4b83-82a7-34aed21fec88 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.301391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039c20c-8712-4b83-82a7-34aed21fec88 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:18.301798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039c20c-8712-4b83-82a7-34aed21fec88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:18.303459 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039c20c-8712-4b83-82a7-34aed21fec88 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:18.330263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 305/623] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 4 09:02:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:18.331940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f04b06-8561-4626-836f-0166330ab7ef None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.333675 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f04b06-8561-4626-836f-0166330ab7ef None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.334240 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f04b06-8561-4626-836f-0166330ab7ef None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.337057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 306/624] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:18.339536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f04b06-8561-4626-836f-0166330ab7ef None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:18.352605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beea58-7721-4885-9b95-eb22e0d840cf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:18.401070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11da4f-061d-4e43-8e38-6ddea5c67459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:18.401914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11da4f-061d-4e43-8e38-6ddea5c67459 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.402230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11da4f-061d-4e43-8e38-6ddea5c67459 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.402510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11da4f-061d-4e43-8e38-6ddea5c67459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.418865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11da4f-061d-4e43-8e38-6ddea5c67459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:18.420654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11da4f-061d-4e43-8e38-6ddea5c67459 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1a11da4f-061d-4e43-8e38-6ddea5c67459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:18.446712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:18.447723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:18.449252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:18.449252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:18.449252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:18.449252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:18.449252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 319/625] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:18.449668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d8def-2025-4945-abf3-909e880c3afb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.449961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d8def-2025-4945-abf3-909e880c3afb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.450245 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d8def-2025-4945-abf3-909e880c3afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.461725 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f04b06-8561-4626-836f-0166330ab7ef None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:18.473594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c78d8def-2025-4945-abf3-909e880c3afb None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:18.474873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c78d8def-2025-4945-abf3-909e880c3afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:18.509466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff96392-77f1-4662-824a-c5f502ddec60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:18.510528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff96392-77f1-4662-824a-c5f502ddec60 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.511234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff96392-77f1-4662-824a-c5f502ddec60 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.512653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff96392-77f1-4662-824a-c5f502ddec60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.557906 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 320/626] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:18.557906 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d8def-2025-4945-abf3-909e880c3afb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:18.591843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:18.599707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:18.618850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff96392-77f1-4662-824a-c5f502ddec60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:18.621141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff96392-77f1-4662-824a-c5f502ddec60 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:18.672665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d8def-2025-4945-abf3-909e880c3afb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:18.712078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0449401-ede3-49e6-ae31-e197a02c2b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:18.712675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0449401-ede3-49e6-ae31-e197a02c2b3b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.712960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0449401-ede3-49e6-ae31-e197a02c2b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.713235 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0449401-ede3-49e6-ae31-e197a02c2b3b None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.714003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0449401-ede3-49e6-ae31-e197a02c2b3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:18.714311 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0449401-ede3-49e6-ae31-e197a02c2b3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:18.715684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0449401-ede3-49e6-ae31-e197a02c2b3b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:18.733899 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 321/627] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:18.734749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0449401-ede3-49e6-ae31-e197a02c2b3b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:18.750264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 307/628] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:02:18.752674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079138c7-253a-40a2-b6ff-6b9f20bfbe03 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.753022 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079138c7-253a-40a2-b6ff-6b9f20bfbe03 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.753364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079138c7-253a-40a2-b6ff-6b9f20bfbe03 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.755172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 308/629] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:18.759509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079138c7-253a-40a2-b6ff-6b9f20bfbe03 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:18.825716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:18.846461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0449401-ede3-49e6-ae31-e197a02c2b3b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:18.852853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:18.870114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079138c7-253a-40a2-b6ff-6b9f20bfbe03 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:18.880515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35107021-8956-4c31-9ded-b893eddefa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:18.881321 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35107021-8956-4c31-9ded-b893eddefa4e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.881599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35107021-8956-4c31-9ded-b893eddefa4e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.881876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35107021-8956-4c31-9ded-b893eddefa4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.892190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35107021-8956-4c31-9ded-b893eddefa4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-35107021-8956-4c31-9ded-b893eddefa4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:18.899921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:18.901101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 322/630] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:18.901991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35107021-8956-4c31-9ded-b893eddefa4e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:18.916705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a91c9d-83ac-4f76-8faf-ac49e72fc062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:18.917503 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91c9d-83ac-4f76-8faf-ac49e72fc062 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.917808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91c9d-83ac-4f76-8faf-ac49e72fc062 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.918115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91c9d-83ac-4f76-8faf-ac49e72fc062 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.925219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a91c9d-83ac-4f76-8faf-ac49e72fc062 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:18.926965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a91c9d-83ac-4f76-8faf-ac49e72fc062 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-00a91c9d-83ac-4f76-8faf-ac49e72fc062 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:18.940451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:18.941636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:18.947469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:18.947469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:18.947469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:18.947469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:18.947469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:18.947469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:18.947469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:18.947469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:18.947469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 309/631] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:18.947469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48571eec-37d0-42f4-8183-b011a8fd6f34 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:18.947469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48571eec-37d0-42f4-8183-b011a8fd6f34 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:18.948093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48571eec-37d0-42f4-8183-b011a8fd6f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:18.960966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-48571eec-37d0-42f4-8183-b011a8fd6f34 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:18.962228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-48571eec-37d0-42f4-8183-b011a8fd6f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:19.004688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35107021-8956-4c31-9ded-b893eddefa4e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:19.029359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 310/632] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:19.030711 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fc4df-2306-4e16-91aa-3bb90f4eba5b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.031003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fc4df-2306-4e16-91aa-3bb90f4eba5b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.031265 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fc4df-2306-4e16-91aa-3bb90f4eba5b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.032512 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 311/633] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:19.033190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fc4df-2306-4e16-91aa-3bb90f4eba5b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:19.045493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1af9b8-cf99-4841-a425-d82c42dbc42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:19.046357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af9b8-cf99-4841-a425-d82c42dbc42b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.046644 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af9b8-cf99-4841-a425-d82c42dbc42b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.046925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1af9b8-cf99-4841-a425-d82c42dbc42b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.058209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1af9b8-cf99-4841-a425-d82c42dbc42b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:19.059683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1af9b8-cf99-4841-a425-d82c42dbc42b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:19.061549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ad1af9b8-cf99-4841-a425-d82c42dbc42b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:19.078163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:19.079265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:19.080335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:19.080335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:19.080335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:19.080335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:19.080335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 323/634] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:19.080678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1af9b8-cf99-4841-a425-d82c42dbc42b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:19.119803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:19.145350 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fc4df-2306-4e16-91aa-3bb90f4eba5b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:19.179724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99567ae0-adfc-47ea-bcb3-c4c119eaefc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:19.180424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99567ae0-adfc-47ea-bcb3-c4c119eaefc8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.180689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99567ae0-adfc-47ea-bcb3-c4c119eaefc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.180968 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99567ae0-adfc-47ea-bcb3-c4c119eaefc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.181717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99567ae0-adfc-47ea-bcb3-c4c119eaefc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:19.182015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1af9b8-cf99-4841-a425-d82c42dbc42b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:19.182123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99567ae0-adfc-47ea-bcb3-c4c119eaefc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:19.183430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99567ae0-adfc-47ea-bcb3-c4c119eaefc8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:19.191453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 312/635] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:19.192652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ba2f6-2dfb-4fa5-9870-3da58982e074 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.192945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ba2f6-2dfb-4fa5-9870-3da58982e074 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.193227 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ba2f6-2dfb-4fa5-9870-3da58982e074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.209368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e24ba2f6-2dfb-4fa5-9870-3da58982e074 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:19.210590 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e24ba2f6-2dfb-4fa5-9870-3da58982e074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:19.216796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5523e-b166-45f8-86db-117237accfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:19.217403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5523e-b166-45f8-86db-117237accfb6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.217685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5523e-b166-45f8-86db-117237accfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.217995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5523e-b166-45f8-86db-117237accfb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.218791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5523e-b166-45f8-86db-117237accfb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:19.219102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5523e-b166-45f8-86db-117237accfb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:19.220533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5523e-b166-45f8-86db-117237accfb6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:19.233278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 324/636] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:19.234105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5523e-b166-45f8-86db-117237accfb6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:19.278936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 313/637] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:19.280081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ba2f6-2dfb-4fa5-9870-3da58982e074 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:19.289241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:19.333312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5523e-b166-45f8-86db-117237accfb6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:19.363694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:19.366736 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e77e2-76c4-4393-a242-56b28162f49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:19.367547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e77e2-76c4-4393-a242-56b28162f49c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.367823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e77e2-76c4-4393-a242-56b28162f49c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.368096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e77e2-76c4-4393-a242-56b28162f49c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.379365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e77e2-76c4-4393-a242-56b28162f49c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-132e77e2-76c4-4393-a242-56b28162f49c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:19.387036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:19.388144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:19.389141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 325/638] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:19.389982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e77e2-76c4-4393-a242-56b28162f49c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:19.399896 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ba2f6-2dfb-4fa5-9870-3da58982e074 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:19.433166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde47ad-cd83-454b-94b9-d75dbdc7b2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:19.433832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde47ad-cd83-454b-94b9-d75dbdc7b2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.434148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde47ad-cd83-454b-94b9-d75dbdc7b2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.434420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde47ad-cd83-454b-94b9-d75dbdc7b2a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.435141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde47ad-cd83-454b-94b9-d75dbdc7b2a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:19.435446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde47ad-cd83-454b-94b9-d75dbdc7b2a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:19.436811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde47ad-cd83-454b-94b9-d75dbdc7b2a4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:19.447909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 314/639] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:19.448687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde47ad-cd83-454b-94b9-d75dbdc7b2a4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:19.493758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e77e2-76c4-4393-a242-56b28162f49c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:19.507492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:19.528118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adbf2b-b69c-4f7b-a9ff-9b573a820f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:19.528919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adbf2b-b69c-4f7b-a9ff-9b573a820f5f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.529192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adbf2b-b69c-4f7b-a9ff-9b573a820f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.529484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adbf2b-b69c-4f7b-a9ff-9b573a820f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.541342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adbf2b-b69c-4f7b-a9ff-9b573a820f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:19.542831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adbf2b-b69c-4f7b-a9ff-9b573a820f5f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:19.552224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde47ad-cd83-454b-94b9-d75dbdc7b2a4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-39adbf2b-b69c-4f7b-a9ff-9b573a820f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:19.559511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:19.560437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:19.561348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:19.561348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:19.561348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:19.561348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:19.565313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 326/640] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:19.566500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adbf2b-b69c-4f7b-a9ff-9b573a820f5f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:19.578381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:19.586623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6556812-71aa-4b09-8dfe-84934a752b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:19.587506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6556812-71aa-4b09-8dfe-84934a752b95 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.587803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6556812-71aa-4b09-8dfe-84934a752b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.588059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6556812-71aa-4b09-8dfe-84934a752b95 None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/8cbb65dbfadf4c63a272f5318efa875a/roles/f0e6d5b558954227a2e1423c3d1e5d18` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.603089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6556812-71aa-4b09-8dfe-84934a752b95 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=8cbb65dbfadf4c63a272f5318efa875a, role_id=f0e6d5b558954227a2e1423c3d1e5d18)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:19.604629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6556812-71aa-4b09-8dfe-84934a752b95 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:19.618662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 315/641] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:02:19 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/8cbb65dbfadf4c63a272f5318efa875a/roles/f0e6d5b558954227a2e1423c3d1e5d18 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:02:19.619453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6556812-71aa-4b09-8dfe-84934a752b95 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:19.687887 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adbf2b-b69c-4f7b-a9ff-9b573a820f5f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:19.727768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12276-3fff-4b4f-ba0a-44b2cf3c12f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:19.728596 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12276-3fff-4b4f-ba0a-44b2cf3c12f1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.728880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12276-3fff-4b4f-ba0a-44b2cf3c12f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.729158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12276-3fff-4b4f-ba0a-44b2cf3c12f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.732973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6556812-71aa-4b09-8dfe-84934a752b95 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:19.740984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12276-3fff-4b4f-ba0a-44b2cf3c12f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cba12276-3fff-4b4f-ba0a-44b2cf3c12f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:19.748996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:19.750268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 327/642] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:19.751148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12276-3fff-4b4f-ba0a-44b2cf3c12f1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:19.783293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5525c1f-9c24-496a-92a3-e9b7e169a8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:19.783983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5525c1f-9c24-496a-92a3-e9b7e169a8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.784261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5525c1f-9c24-496a-92a3-e9b7e169a8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.784540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5525c1f-9c24-496a-92a3-e9b7e169a8f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.785451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5525c1f-9c24-496a-92a3-e9b7e169a8f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:19.785767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5525c1f-9c24-496a-92a3-e9b7e169a8f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:19.787198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5525c1f-9c24-496a-92a3-e9b7e169a8f5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:19.789731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:19.796633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 316/643] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:19.797506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5525c1f-9c24-496a-92a3-e9b7e169a8f5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:19.842964 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:19.873399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12276-3fff-4b4f-ba0a-44b2cf3c12f1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:19.909006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5525c1f-9c24-496a-92a3-e9b7e169a8f5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:19.909932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fd9a2-28f8-4d27-8f38-d0c014f2f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:19.910787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd9a2-28f8-4d27-8f38-d0c014f2f1cb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.911102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd9a2-28f8-4d27-8f38-d0c014f2f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.911379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd9a2-28f8-4d27-8f38-d0c014f2f1cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.918350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fd9a2-28f8-4d27-8f38-d0c014f2f1cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:19.919822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fd9a2-28f8-4d27-8f38-d0c014f2f1cb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8e0fd9a2-28f8-4d27-8f38-d0c014f2f1cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:19.935023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:19.936003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:19.936974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:19.936974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:19.936974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:19.936974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:19.936974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:19.936974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:19.936974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:19.936974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:19.936974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 328/644] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:19.938491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fd9a2-28f8-4d27-8f38-d0c014f2f1cb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:19.951958 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a45ae-b5c2-4a43-b86f-457405735f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:19.952689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a45ae-b5c2-4a43-b86f-457405735f7f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:19.952975 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a45ae-b5c2-4a43-b86f-457405735f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:19.953307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a45ae-b5c2-4a43-b86f-457405735f7f None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:19.954081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a45ae-b5c2-4a43-b86f-457405735f7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:19.954439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a45ae-b5c2-4a43-b86f-457405735f7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:19.955872 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a45ae-b5c2-4a43-b86f-457405735f7f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:19.972994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 317/645] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:19.972994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a45ae-b5c2-4a43-b86f-457405735f7f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:20.031515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:20.051506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fd9a2-28f8-4d27-8f38-d0c014f2f1cb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:20.083546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:20.087013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a45ae-b5c2-4a43-b86f-457405735f7f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:20.091781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde26530-2e85-477e-b51f-0d27285b86ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:20.092537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde26530-2e85-477e-b51f-0d27285b86ce None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:20.092919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde26530-2e85-477e-b51f-0d27285b86ce None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:20.093262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde26530-2e85-477e-b51f-0d27285b86ce None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:20.094160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde26530-2e85-477e-b51f-0d27285b86ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:20.094535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde26530-2e85-477e-b51f-0d27285b86ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:20.095994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde26530-2e85-477e-b51f-0d27285b86ce None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:20.109425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 329/646] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:20.109425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde26530-2e85-477e-b51f-0d27285b86ce None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:20.134374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83426d2c-aa31-4a53-a6bb-a24d08756b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:20.135437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83426d2c-aa31-4a53-a6bb-a24d08756b16 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:20.135723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83426d2c-aa31-4a53-a6bb-a24d08756b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:20.136001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83426d2c-aa31-4a53-a6bb-a24d08756b16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:20.146728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83426d2c-aa31-4a53-a6bb-a24d08756b16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:20.148189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83426d2c-aa31-4a53-a6bb-a24d08756b16 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-83426d2c-aa31-4a53-a6bb-a24d08756b16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:20.169237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:20.170231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:20.171156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:20.171156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:20.171156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:20.171156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:20.171156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 318/647] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:20.171450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83426d2c-aa31-4a53-a6bb-a24d08756b16 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:20.218027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde26530-2e85-477e-b51f-0d27285b86ce None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:20.259923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb075c-01a7-4903-aafe-cd632a04097a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:20.261414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb075c-01a7-4903-aafe-cd632a04097a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:20.261833 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb075c-01a7-4903-aafe-cd632a04097a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:20.262233 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb075c-01a7-4903-aafe-cd632a04097a None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:20.263356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb075c-01a7-4903-aafe-cd632a04097a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:20.263763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb075c-01a7-4903-aafe-cd632a04097a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:20.265655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb075c-01a7-4903-aafe-cd632a04097a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:20.273863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83426d2c-aa31-4a53-a6bb-a24d08756b16 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:20.287118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 330/648] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:20.287118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb075c-01a7-4903-aafe-cd632a04097a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:20.298958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:20.318479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2cc7d-0ce1-4897-a889-ba8214690f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:20.319324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2cc7d-0ce1-4897-a889-ba8214690f22 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:20.319632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2cc7d-0ce1-4897-a889-ba8214690f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:20.319900 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2cc7d-0ce1-4897-a889-ba8214690f22 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:20.328130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2cc7d-0ce1-4897-a889-ba8214690f22 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:20.329691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2cc7d-0ce1-4897-a889-ba8214690f22 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-9aa2cc7d-0ce1-4897-a889-ba8214690f22 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:20.344434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:20.345666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:20.346863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:20.346863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:20.346863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:20.346863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:20.346863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:20.346863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:20.346863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:20.346863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:20.346863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:02:20.346863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:20.347423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 319/649] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:20.348024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2cc7d-0ce1-4897-a889-ba8214690f22 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:20.363572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:20.402413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb075c-01a7-4903-aafe-cd632a04097a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:20.437257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00608b13-fe09-4e41-9f4d-9220b2a95e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:20.438384 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00608b13-fe09-4e41-9f4d-9220b2a95e82 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:20.438674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00608b13-fe09-4e41-9f4d-9220b2a95e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:20.438956 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00608b13-fe09-4e41-9f4d-9220b2a95e82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:20.449322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00608b13-fe09-4e41-9f4d-9220b2a95e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:20.450895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00608b13-fe09-4e41-9f4d-9220b2a95e82 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:20.460870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2cc7d-0ce1-4897-a889-ba8214690f22 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-00608b13-fe09-4e41-9f4d-9220b2a95e82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:20.468553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:20.469635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:20.470651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:20.470651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:20.470651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:20.470651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:20.470651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 331/650] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:20.470931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00608b13-fe09-4e41-9f4d-9220b2a95e82 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:20.501926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2e0f2-9354-48d8-86e2-94366045391c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:20.503200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2e0f2-9354-48d8-86e2-94366045391c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:20.503660 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2e0f2-9354-48d8-86e2-94366045391c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:20.504378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2e0f2-9354-48d8-86e2-94366045391c None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:20.504974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2e0f2-9354-48d8-86e2-94366045391c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:20.505185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2e0f2-9354-48d8-86e2-94366045391c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:20.506603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2e0f2-9354-48d8-86e2-94366045391c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:20.526116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 320/651] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:20.526116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2e0f2-9354-48d8-86e2-94366045391c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:20.541903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:20.582821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00608b13-fe09-4e41-9f4d-9220b2a95e82 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:20.600640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:20.619642 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69f6a3-2cdd-453e-8424-37f055c831f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:20.620934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69f6a3-2cdd-453e-8424-37f055c831f5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:20.621371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69f6a3-2cdd-453e-8424-37f055c831f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:20.621794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69f6a3-2cdd-453e-8424-37f055c831f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:20.622832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69f6a3-2cdd-453e-8424-37f055c831f5 None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=739f3b5eaef542c29a0cf0b475c0d965) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:20.623306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69f6a3-2cdd-453e-8424-37f055c831f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:20.624767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69f6a3-2cdd-453e-8424-37f055c831f5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:20.641911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 332/652] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:20 2022] GET /identity/v3/role_assignments?user.id=739f3b5eaef542c29a0cf0b475c0d965&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:20.641911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69f6a3-2cdd-453e-8424-37f055c831f5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:20.642235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2e0f2-9354-48d8-86e2-94366045391c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:20.692517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958e6ba-901b-464e-8d87-e48f229ecba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:20.693253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958e6ba-901b-464e-8d87-e48f229ecba2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:20.693541 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958e6ba-901b-464e-8d87-e48f229ecba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:20.693820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958e6ba-901b-464e-8d87-e48f229ecba2 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:20.694739 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958e6ba-901b-464e-8d87-e48f229ecba2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:20.695064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958e6ba-901b-464e-8d87-e48f229ecba2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:20.696567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958e6ba-901b-464e-8d87-e48f229ecba2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:20.723654 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 321/653] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 4 09:02:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:20.724858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958e6ba-901b-464e-8d87-e48f229ecba2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:20.789737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69f6a3-2cdd-453e-8424-37f055c831f5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:20.817328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:20.835521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e8533-c55b-4d6f-9741-44bc53ae779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:20.836205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e8533-c55b-4d6f-9741-44bc53ae779c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:20.836757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e8533-c55b-4d6f-9741-44bc53ae779c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:20.837041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e8533-c55b-4d6f-9741-44bc53ae779c None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:20.838047 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e8533-c55b-4d6f-9741-44bc53ae779c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:20.838359 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e8533-c55b-4d6f-9741-44bc53ae779c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:20.840230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e8533-c55b-4d6f-9741-44bc53ae779c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:20.845754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958e6ba-901b-464e-8d87-e48f229ecba2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:20.853645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 333/654] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:20.854519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e8533-c55b-4d6f-9741-44bc53ae779c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:20.892955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:20.902342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116d363-6797-4aff-943b-97a65ab72fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:20.903156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d363-6797-4aff-943b-97a65ab72fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:20.903434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d363-6797-4aff-943b-97a65ab72fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:20.903705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d363-6797-4aff-943b-97a65ab72fd2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:20.912463 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116d363-6797-4aff-943b-97a65ab72fd2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:20.913929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116d363-6797-4aff-943b-97a65ab72fd2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-9116d363-6797-4aff-943b-97a65ab72fd2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:20.931104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:20.932321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 322/655] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:20.933444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116d363-6797-4aff-943b-97a65ab72fd2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:20.988053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e8533-c55b-4d6f-9741-44bc53ae779c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:21.036436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ad84a-58d5-4a23-8aad-9a9da566a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:21.037916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ad84a-58d5-4a23-8aad-9a9da566a62d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:21.038521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ad84a-58d5-4a23-8aad-9a9da566a62d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:21.039058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ad84a-58d5-4a23-8aad-9a9da566a62d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:21.048957 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116d363-6797-4aff-943b-97a65ab72fd2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:21.085963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1190fa-9555-4f61-aee8-e3ed4854697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:21.087204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1190fa-9555-4f61-aee8-e3ed4854697a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:21.087711 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1190fa-9555-4f61-aee8-e3ed4854697a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:21.088188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1190fa-9555-4f61-aee8-e3ed4854697a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:21.096406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1190fa-9555-4f61-aee8-e3ed4854697a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:21.098223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1190fa-9555-4f61-aee8-e3ed4854697a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:21.108758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-4b1190fa-9555-4f61-aee8-e3ed4854697a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:21.114509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:21.115523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:21.116583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:21.116583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:21.116583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:21.116583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:21.116583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:21.116583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:21.116583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:21.116583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:21.116583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 04 09:02:21.116583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:21.123618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 323/656] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:21.124735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1190fa-9555-4f61-aee8-e3ed4854697a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:21.130771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ad84a-58d5-4a23-8aad-9a9da566a62d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:21.135512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ad84a-58d5-4a23-8aad-9a9da566a62d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:21.184974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:21.241927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1190fa-9555-4f61-aee8-e3ed4854697a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:21.277607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 334/657] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:02:21.278869 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ad84a-58d5-4a23-8aad-9a9da566a62d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:21.282843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025a1f5-49bb-49b5-8fba-c33767a11790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:21.283012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025a1f5-49bb-49b5-8fba-c33767a11790 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:21.283335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025a1f5-49bb-49b5-8fba-c33767a11790 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:21.283709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025a1f5-49bb-49b5-8fba-c33767a11790 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:21.284646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025a1f5-49bb-49b5-8fba-c33767a11790 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:21.285067 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025a1f5-49bb-49b5-8fba-c33767a11790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:21.286661 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025a1f5-49bb-49b5-8fba-c33767a11790 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:21.310909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 324/658] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 4 09:02:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:21.312095 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025a1f5-49bb-49b5-8fba-c33767a11790 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:21.367821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:21.384992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ad84a-58d5-4a23-8aad-9a9da566a62d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:21.416764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025a1f5-49bb-49b5-8fba-c33767a11790 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:21.421809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df21ab-fa80-4e1d-9fc3-e697f1616578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:21.423161 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df21ab-fa80-4e1d-9fc3-e697f1616578 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:21.423658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df21ab-fa80-4e1d-9fc3-e697f1616578 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:21.424145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df21ab-fa80-4e1d-9fc3-e697f1616578 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:21.425200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df21ab-fa80-4e1d-9fc3-e697f1616578 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:21.425827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df21ab-fa80-4e1d-9fc3-e697f1616578 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:21.427630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df21ab-fa80-4e1d-9fc3-e697f1616578 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:21.435860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:21.446895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 335/659] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jul 4 09:02:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:21.448016 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df21ab-fa80-4e1d-9fc3-e697f1616578 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:21.461382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f1b43-25fe-4879-b0b0-5f52990dc670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:21.462309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f1b43-25fe-4879-b0b0-5f52990dc670 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:21.462600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f1b43-25fe-4879-b0b0-5f52990dc670 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:21.462876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f1b43-25fe-4879-b0b0-5f52990dc670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:21.474673 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f1b43-25fe-4879-b0b0-5f52990dc670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:21.476269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f1b43-25fe-4879-b0b0-5f52990dc670 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-178f1b43-25fe-4879-b0b0-5f52990dc670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:21.496237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:21.497245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:21.498212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:21.498212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:21.498212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:21.498212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:21.502459 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 325/660] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:21.503413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f1b43-25fe-4879-b0b0-5f52990dc670 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:21.561341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df21ab-fa80-4e1d-9fc3-e697f1616578 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:21.597775 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f6825-5969-485c-8e2b-0b2b0cac7e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:21.598847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f6825-5969-485c-8e2b-0b2b0cac7e63 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:21.599502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f6825-5969-485c-8e2b-0b2b0cac7e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:21.600001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f6825-5969-485c-8e2b-0b2b0cac7e63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:21.612612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:21.613705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f1b43-25fe-4879-b0b0-5f52990dc670 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:21.649758 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3c0f7-6c77-41db-8f82-306d926a777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:21.650382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c0f7-6c77-41db-8f82-306d926a777b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:21.650667 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c0f7-6c77-41db-8f82-306d926a777b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:21.650946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c0f7-6c77-41db-8f82-306d926a777b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:21.671027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f6825-5969-485c-8e2b-0b2b0cac7e63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:21.672549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f6825-5969-485c-8e2b-0b2b0cac7e63 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:21.680785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:21.726728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3c0f7-6c77-41db-8f82-306d926a777b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:21.728623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3c0f7-6c77-41db-8f82-306d926a777b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:21.771666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 336/661] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:02:21.776812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f6825-5969-485c-8e2b-0b2b0cac7e63 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:21.818503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:21.836018 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 326/662] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:02:21.837642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48570a7-509d-4aae-a056-5e3af684eed1 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:21.838142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48570a7-509d-4aae-a056-5e3af684eed1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:21.838631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48570a7-509d-4aae-a056-5e3af684eed1 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:21.844230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 327/663] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:21.845216 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48570a7-509d-4aae-a056-5e3af684eed1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:21.884207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f6825-5969-485c-8e2b-0b2b0cac7e63 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:21.911486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:21.923040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a41cc-69fc-4ee0-bdf7-586c1abcc5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:21.923734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a41cc-69fc-4ee0-bdf7-586c1abcc5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:21.924016 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a41cc-69fc-4ee0-bdf7-586c1abcc5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:21.924283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a41cc-69fc-4ee0-bdf7-586c1abcc5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:21.925004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a41cc-69fc-4ee0-bdf7-586c1abcc5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:21.925307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a41cc-69fc-4ee0-bdf7-586c1abcc5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:21.926857 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a41cc-69fc-4ee0-bdf7-586c1abcc5c3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:21.943461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 337/664] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:21.943461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a41cc-69fc-4ee0-bdf7-586c1abcc5c3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:21.961564 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48570a7-509d-4aae-a056-5e3af684eed1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:21.997785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db389b7-1fd1-4ded-a72c-9c2b5c6c225e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:21.998720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db389b7-1fd1-4ded-a72c-9c2b5c6c225e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:21.999072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db389b7-1fd1-4ded-a72c-9c2b5c6c225e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:21.999303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db389b7-1fd1-4ded-a72c-9c2b5c6c225e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.010537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db389b7-1fd1-4ded-a72c-9c2b5c6c225e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:22.011979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db389b7-1fd1-4ded-a72c-9c2b5c6c225e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-8db389b7-1fd1-4ded-a72c-9c2b5c6c225e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:22.030362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:22.031285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:22.032238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:22.032238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:22.032238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:22.032238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:22.032238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 328/665] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:22.032989 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a920611-b24f-4edb-8c0a-6efbed22b6ce None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.033156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a920611-b24f-4edb-8c0a-6efbed22b6ce None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.033399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a920611-b24f-4edb-8c0a-6efbed22b6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.045316 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a41cc-69fc-4ee0-bdf7-586c1abcc5c3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:22.047052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:22.053429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3a920611-b24f-4edb-8c0a-6efbed22b6ce None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:22.054652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3a920611-b24f-4edb-8c0a-6efbed22b6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:22.084403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e873ac-7e23-477c-9aff-1ab26b3a3e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:22.085204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e873ac-7e23-477c-9aff-1ab26b3a3e4e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.085485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e873ac-7e23-477c-9aff-1ab26b3a3e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.085755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e873ac-7e23-477c-9aff-1ab26b3a3e4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.097010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e873ac-7e23-477c-9aff-1ab26b3a3e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:22.098680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e873ac-7e23-477c-9aff-1ab26b3a3e4e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e4e873ac-7e23-477c-9aff-1ab26b3a3e4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:22.116269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:22.117676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:22.119150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:22.119150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:22.119150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:22.119150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:22.119150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 338/666] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:22.119150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e873ac-7e23-477c-9aff-1ab26b3a3e4e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:22.134584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 329/667] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:22.135518 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a920611-b24f-4edb-8c0a-6efbed22b6ce None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:22.183741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:22.245663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e873ac-7e23-477c-9aff-1ab26b3a3e4e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:22.245935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a920611-b24f-4edb-8c0a-6efbed22b6ce None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:22.282929 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc74350-1300-4c2d-be47-8d8c4e3d4129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:22.283789 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc74350-1300-4c2d-be47-8d8c4e3d4129 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.284065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc74350-1300-4c2d-be47-8d8c4e3d4129 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.284350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc74350-1300-4c2d-be47-8d8c4e3d4129 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.287159 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9ffe-086d-4c6b-8bfb-e89e3849249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:22.287897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9ffe-086d-4c6b-8bfb-e89e3849249d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.288258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9ffe-086d-4c6b-8bfb-e89e3849249d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.288595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9ffe-086d-4c6b-8bfb-e89e3849249d None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.289414 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc9ffe-086d-4c6b-8bfb-e89e3849249d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:22.289791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc9ffe-086d-4c6b-8bfb-e89e3849249d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:22.291249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc9ffe-086d-4c6b-8bfb-e89e3849249d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:22.291861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc74350-1300-4c2d-be47-8d8c4e3d4129 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:22.293327 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc74350-1300-4c2d-be47-8d8c4e3d4129 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:22.307568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 330/668] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:22.307568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9ffe-086d-4c6b-8bfb-e89e3849249d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-0bc74350-1300-4c2d-be47-8d8c4e3d4129 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:22.307962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:22.308963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:22.310315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:22.310315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:22.310315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:22.310315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:22.310315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:22.310315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:22.310315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:22.310315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:22.310315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 339/669] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:22.310315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc74350-1300-4c2d-be47-8d8c4e3d4129 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:22.329431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:22.417005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9ffe-086d-4c6b-8bfb-e89e3849249d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:22.418744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:22.419335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc74350-1300-4c2d-be47-8d8c4e3d4129 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:22.459380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f0ebb-7b13-4682-8f43-3b770c13dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:22.460233 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f0ebb-7b13-4682-8f43-3b770c13dab8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.460534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f0ebb-7b13-4682-8f43-3b770c13dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.460806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f0ebb-7b13-4682-8f43-3b770c13dab8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.461416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006ca25-f74b-46ff-83f0-74d4e01dbc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:22.462151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006ca25-f74b-46ff-83f0-74d4e01dbc44 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.462440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006ca25-f74b-46ff-83f0-74d4e01dbc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.462715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006ca25-f74b-46ff-83f0-74d4e01dbc44 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.463473 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006ca25-f74b-46ff-83f0-74d4e01dbc44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:22.463791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006ca25-f74b-46ff-83f0-74d4e01dbc44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:22.465250 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006ca25-f74b-46ff-83f0-74d4e01dbc44 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:22.473480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f0ebb-7b13-4682-8f43-3b770c13dab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:22.482069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 331/670] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:22.482069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006ca25-f74b-46ff-83f0-74d4e01dbc44 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ec8f0ebb-7b13-4682-8f43-3b770c13dab8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:22.482331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:22.483484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 340/671] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:22.484341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f0ebb-7b13-4682-8f43-3b770c13dab8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:22.565212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:22.588888 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006ca25-f74b-46ff-83f0-74d4e01dbc44 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:22.593724 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f0ebb-7b13-4682-8f43-3b770c13dab8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:22.623121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5275a26-4bdb-4ae9-a6bc-3d6812605f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:22.623923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5275a26-4bdb-4ae9-a6bc-3d6812605f4a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.624206 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5275a26-4bdb-4ae9-a6bc-3d6812605f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.624479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5275a26-4bdb-4ae9-a6bc-3d6812605f4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.632142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5275a26-4bdb-4ae9-a6bc-3d6812605f4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:22.634004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5275a26-4bdb-4ae9-a6bc-3d6812605f4a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:22.635259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a0b35-7f7a-4245-ac52-266bb303d6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:22.636092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a0b35-7f7a-4245-ac52-266bb303d6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.636379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a0b35-7f7a-4245-ac52-266bb303d6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.636942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a0b35-7f7a-4245-ac52-266bb303d6f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.637737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a0b35-7f7a-4245-ac52-266bb303d6f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:22.638093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a0b35-7f7a-4245-ac52-266bb303d6f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:22.639545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a0b35-7f7a-4245-ac52-266bb303d6f3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c5275a26-4bdb-4ae9-a6bc-3d6812605f4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:22.650441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:22.651357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:22.652832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:22.652832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:22.652832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:22.652832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:22.652832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:22.652832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:22.652832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:22.652832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:22.652832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 332/672] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:22.652832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5275a26-4bdb-4ae9-a6bc-3d6812605f4a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:22.654177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 341/673] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:22.655543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a0b35-7f7a-4245-ac52-266bb303d6f3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:22.680763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:22.760397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5275a26-4bdb-4ae9-a6bc-3d6812605f4a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:22.762923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a0b35-7f7a-4245-ac52-266bb303d6f3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:22.797121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcadd9b-1ddb-4577-af12-e562994e902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:22.797967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcadd9b-1ddb-4577-af12-e562994e902f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.798255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcadd9b-1ddb-4577-af12-e562994e902f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.798663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcadd9b-1ddb-4577-af12-e562994e902f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.803521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f5ce4-7282-4e5f-9cc0-f668609e01bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:22.804202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f5ce4-7282-4e5f-9cc0-f668609e01bf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.804488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f5ce4-7282-4e5f-9cc0-f668609e01bf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.804773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f5ce4-7282-4e5f-9cc0-f668609e01bf None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.805528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f5ce4-7282-4e5f-9cc0-f668609e01bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:22.805695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcadd9b-1ddb-4577-af12-e562994e902f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:22.805854 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f5ce4-7282-4e5f-9cc0-f668609e01bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:22.807224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcadd9b-1ddb-4577-af12-e562994e902f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:22.807320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f5ce4-7282-4e5f-9cc0-f668609e01bf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:22.808897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:22.821217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 342/674] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:22.821217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f5ce4-7282-4e5f-9cc0-f668609e01bf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ddcadd9b-1ddb-4577-af12-e562994e902f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:22.821871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:22.822774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:22.823679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:22.823679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:22.823679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:22.823679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:22.823679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:22.823679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:22.823679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:22.823679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:22.823679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 333/675] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:22.824169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcadd9b-1ddb-4577-af12-e562994e902f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:22.904495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:22.921172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f5ce4-7282-4e5f-9cc0-f668609e01bf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:22.923211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcadd9b-1ddb-4577-af12-e562994e902f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:22.953739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0bac9-e813-48fe-9e77-d46ab48bedb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:22.954517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0bac9-e813-48fe-9e77-d46ab48bedb4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.954808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0bac9-e813-48fe-9e77-d46ab48bedb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.955071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0bac9-e813-48fe-9e77-d46ab48bedb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.955857 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0bac9-e813-48fe-9e77-d46ab48bedb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:22.956167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0bac9-e813-48fe-9e77-d46ab48bedb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:22.956301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcde9cc-691f-4a08-b583-825cf06513d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:22.956921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcde9cc-691f-4a08-b583-825cf06513d1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.957200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcde9cc-691f-4a08-b583-825cf06513d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.957475 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcde9cc-691f-4a08-b583-825cf06513d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.957622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0bac9-e813-48fe-9e77-d46ab48bedb4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:22.958342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcde9cc-691f-4a08-b583-825cf06513d1 None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=e9f34c35a9ba4376a9b55ec5e23cf35f) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:22.958656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcde9cc-691f-4a08-b583-825cf06513d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:22.960068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcde9cc-691f-4a08-b583-825cf06513d1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:22.965859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 334/676] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/role_assignments?user.id=e9f34c35a9ba4376a9b55ec5e23cf35f&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:22.966656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcde9cc-691f-4a08-b583-825cf06513d1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:22.978971 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 343/677] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:22.981272 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2bb0e-95f3-4416-8d30-1d08769c8bb4 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:22.981774 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2bb0e-95f3-4416-8d30-1d08769c8bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:22.982365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2bb0e-95f3-4416-8d30-1d08769c8bb4 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:22.984863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 344/678] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:22.986108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2bb0e-95f3-4416-8d30-1d08769c8bb4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:23.053649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:23.081003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcde9cc-691f-4a08-b583-825cf06513d1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:23.106038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2bb0e-95f3-4416-8d30-1d08769c8bb4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:23.123603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c849cc-c3e5-4919-a1d7-07146fc2b847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:23.124420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c849cc-c3e5-4919-a1d7-07146fc2b847 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:23.124708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c849cc-c3e5-4919-a1d7-07146fc2b847 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:23.124992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c849cc-c3e5-4919-a1d7-07146fc2b847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:23.138497 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c849cc-c3e5-4919-a1d7-07146fc2b847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:23.139949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c849cc-c3e5-4919-a1d7-07146fc2b847 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:23.149086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:23.154563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c2ea4-1bb9-4035-86e9-bb662e5fee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:23.155477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c2ea4-1bb9-4035-86e9-bb662e5fee33 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:23.155895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c2ea4-1bb9-4035-86e9-bb662e5fee33 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:23.156314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c2ea4-1bb9-4035-86e9-bb662e5fee33 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:23.157270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c2ea4-1bb9-4035-86e9-bb662e5fee33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:23.157737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c2ea4-1bb9-4035-86e9-bb662e5fee33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:23.159810 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c2ea4-1bb9-4035-86e9-bb662e5fee33 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-15c849cc-c3e5-4919-a1d7-07146fc2b847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:23.162754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:23.163712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:23.164816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:23.164816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:23.164816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:23.164816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:23.166066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 335/679] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:23.166939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c849cc-c3e5-4919-a1d7-07146fc2b847 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:23.177261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 345/680] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:23.178827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67d3d-68af-4053-8ea7-9a4063fd7de7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:23.179119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67d3d-68af-4053-8ea7-9a4063fd7de7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:23.179398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67d3d-68af-4053-8ea7-9a4063fd7de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:23.198320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-52e67d3d-68af-4053-8ea7-9a4063fd7de7 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:23.199522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-52e67d3d-68af-4053-8ea7-9a4063fd7de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:23.271420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 346/681] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:23.272531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e67d3d-68af-4053-8ea7-9a4063fd7de7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:23.289944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c849cc-c3e5-4919-a1d7-07146fc2b847 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:23.320098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:23.328406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46e959-3d8d-49f7-ad72-b74576626ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:23.329182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e959-3d8d-49f7-ad72-b74576626ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:23.329582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e959-3d8d-49f7-ad72-b74576626ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:23.329986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e959-3d8d-49f7-ad72-b74576626ab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:23.330921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46e959-3d8d-49f7-ad72-b74576626ab1 None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=107a90e384504e8d80264d6bf20f1d9b) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:23.331333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46e959-3d8d-49f7-ad72-b74576626ab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:23.333182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46e959-3d8d-49f7-ad72-b74576626ab1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:23.349928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 336/682] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:23 2022] GET /identity/v3/role_assignments?user.id=107a90e384504e8d80264d6bf20f1d9b&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:23.349928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46e959-3d8d-49f7-ad72-b74576626ab1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:23.397241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e67d3d-68af-4053-8ea7-9a4063fd7de7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:23.432385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:23.437628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601976b0-2a8d-47bc-a180-2348de39b541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:23.438754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601976b0-2a8d-47bc-a180-2348de39b541 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:23.439105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601976b0-2a8d-47bc-a180-2348de39b541 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:23.439445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601976b0-2a8d-47bc-a180-2348de39b541 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:23.440275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601976b0-2a8d-47bc-a180-2348de39b541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:23.440702 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601976b0-2a8d-47bc-a180-2348de39b541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:23.442477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601976b0-2a8d-47bc-a180-2348de39b541 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:23.461230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 347/683] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:23.462335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601976b0-2a8d-47bc-a180-2348de39b541 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:23.485154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46e959-3d8d-49f7-ad72-b74576626ab1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:23.530498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a5983-7908-4894-99c8-e254c0186aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:23.531168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a5983-7908-4894-99c8-e254c0186aab None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:23.531454 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a5983-7908-4894-99c8-e254c0186aab None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:23.531732 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a5983-7908-4894-99c8-e254c0186aab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:23.533937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a5983-7908-4894-99c8-e254c0186aab None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=98649aa38f764af0b41ff5594b94a989) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:23.533937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a5983-7908-4894-99c8-e254c0186aab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:23.534378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a5983-7908-4894-99c8-e254c0186aab None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:23.546085 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 337/684] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:23 2022] GET /identity/v3/role_assignments?user.id=98649aa38f764af0b41ff5594b94a989&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:23.547050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a5983-7908-4894-99c8-e254c0186aab None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:23.597180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601976b0-2a8d-47bc-a180-2348de39b541 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:23.607386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:23.655637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556838a8-5ece-42e1-a379-3bfc76feb759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:23.656469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556838a8-5ece-42e1-a379-3bfc76feb759 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:23.656753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556838a8-5ece-42e1-a379-3bfc76feb759 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:23.657032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556838a8-5ece-42e1-a379-3bfc76feb759 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:23.671554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556838a8-5ece-42e1-a379-3bfc76feb759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-556838a8-5ece-42e1-a379-3bfc76feb759 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:23.680065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:23.681124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:23.682037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 348/685] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:23.687665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556838a8-5ece-42e1-a379-3bfc76feb759 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:23.701072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a5983-7908-4894-99c8-e254c0186aab None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:23.735340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:23.751478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbfbff-df4a-4fdf-972d-e9d37a9a0711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:23.751478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbfbff-df4a-4fdf-972d-e9d37a9a0711 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:23.751478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbfbff-df4a-4fdf-972d-e9d37a9a0711 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:23.751478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbfbff-df4a-4fdf-972d-e9d37a9a0711 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:23.760664 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bbfbff-df4a-4fdf-972d-e9d37a9a0711 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:23.762264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bbfbff-df4a-4fdf-972d-e9d37a9a0711 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-75bbfbff-df4a-4fdf-972d-e9d37a9a0711 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:23.778110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:23.779161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:23.780135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:23.780135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:23.780135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:23.780135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:23.780135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:23.780135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:23.780135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:23.780135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:23.780135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:23.780135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:23.780135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 338/686] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:23.785525 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbfbff-df4a-4fdf-972d-e9d37a9a0711 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:23.811124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556838a8-5ece-42e1-a379-3bfc76feb759 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:23.846570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c264f5-fbe9-4ee8-84e8-19f4bdbb76a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:23.847310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c264f5-fbe9-4ee8-84e8-19f4bdbb76a3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:23.847640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c264f5-fbe9-4ee8-84e8-19f4bdbb76a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:23.847919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c264f5-fbe9-4ee8-84e8-19f4bdbb76a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:23.848704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c264f5-fbe9-4ee8-84e8-19f4bdbb76a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:23.849021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c264f5-fbe9-4ee8-84e8-19f4bdbb76a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:23.850514 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c264f5-fbe9-4ee8-84e8-19f4bdbb76a3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:23.859979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 349/687] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:23.865864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c264f5-fbe9-4ee8-84e8-19f4bdbb76a3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:23.896513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:23.898498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbfbff-df4a-4fdf-972d-e9d37a9a0711 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:23.936110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f28e03-40fa-4518-9126-bf8c48cd8117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:23.936784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f28e03-40fa-4518-9126-bf8c48cd8117 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:23.937163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f28e03-40fa-4518-9126-bf8c48cd8117 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:23.937447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f28e03-40fa-4518-9126-bf8c48cd8117 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:23.938443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f28e03-40fa-4518-9126-bf8c48cd8117 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:23.938759 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f28e03-40fa-4518-9126-bf8c48cd8117 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:23.941084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f28e03-40fa-4518-9126-bf8c48cd8117 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:23.960129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 339/688] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 4 09:02:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:23.965666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f28e03-40fa-4518-9126-bf8c48cd8117 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:23.974713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c264f5-fbe9-4ee8-84e8-19f4bdbb76a3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:24.010197 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af05be2-004e-48c9-86b6-1cfa90183029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:24.011013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05be2-004e-48c9-86b6-1cfa90183029 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:24.011295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05be2-004e-48c9-86b6-1cfa90183029 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:24.011570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05be2-004e-48c9-86b6-1cfa90183029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:24.022921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af05be2-004e-48c9-86b6-1cfa90183029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:24.023604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:24.024415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af05be2-004e-48c9-86b6-1cfa90183029 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2af05be2-004e-48c9-86b6-1cfa90183029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:24.043239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:24.044219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:24.045213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:24.045213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:24.045213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:24.045213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:24.045213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 350/689] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:24.050722 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af05be2-004e-48c9-86b6-1cfa90183029 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:24.073832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f28e03-40fa-4518-9126-bf8c48cd8117 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:24.111156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55e697-6bf0-42db-9201-8f6c5571cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:24.111762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55e697-6bf0-42db-9201-8f6c5571cb8c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:24.112053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55e697-6bf0-42db-9201-8f6c5571cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:24.112316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55e697-6bf0-42db-9201-8f6c5571cb8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:24.148649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:24.177771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af05be2-004e-48c9-86b6-1cfa90183029 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:24.239084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96daac3-f3af-49b3-8231-bb916c22d23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:24.239765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96daac3-f3af-49b3-8231-bb916c22d23f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:24.240046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96daac3-f3af-49b3-8231-bb916c22d23f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:24.240319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96daac3-f3af-49b3-8231-bb916c22d23f None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:24.241055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96daac3-f3af-49b3-8231-bb916c22d23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:24.241361 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96daac3-f3af-49b3-8231-bb916c22d23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:24.242783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96daac3-f3af-49b3-8231-bb916c22d23f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:24.265798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55e697-6bf0-42db-9201-8f6c5571cb8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:24.267328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55e697-6bf0-42db-9201-8f6c5571cb8c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:24.276446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 351/690] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:24.287017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96daac3-f3af-49b3-8231-bb916c22d23f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:24.332132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:24.407009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 340/691] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:02:24.412601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55e697-6bf0-42db-9201-8f6c5571cb8c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:24.451356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96daac3-f3af-49b3-8231-bb916c22d23f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:24.516054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55e697-6bf0-42db-9201-8f6c5571cb8c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:24.517965 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2e490-7efb-4158-9d0c-f5c750fba70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:24.519399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2e490-7efb-4158-9d0c-f5c750fba70f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:24.519826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2e490-7efb-4158-9d0c-f5c750fba70f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:24.520241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2e490-7efb-4158-9d0c-f5c750fba70f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:24.535301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2e490-7efb-4158-9d0c-f5c750fba70f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:24.537150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2e490-7efb-4158-9d0c-f5c750fba70f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:24.543711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:24.551835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddf2b1-a35c-4e73-9874-8c48df6540c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:24.552829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddf2b1-a35c-4e73-9874-8c48df6540c2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:24.553288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddf2b1-a35c-4e73-9874-8c48df6540c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:24.553709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddf2b1-a35c-4e73-9874-8c48df6540c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-36b2e490-7efb-4158-9d0c-f5c750fba70f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:24.562489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:24.563706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:24.564900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:24.564900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:24.564900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:24.564900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:24.564900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:24.564900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:24.564900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:24.564900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:24.564900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:24.564900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:24.567622 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddf2b1-a35c-4e73-9874-8c48df6540c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:24.569285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddf2b1-a35c-4e73-9874-8c48df6540c2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:24.569835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 352/692] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 4 09:02:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:24.572294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:24.577661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2e490-7efb-4158-9d0c-f5c750fba70f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-04ddf2b1-a35c-4e73-9874-8c48df6540c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:24.595879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:24.597005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:24.598084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:24.598084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:24.598084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:24.598084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:24.598326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 341/693] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:24.604069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddf2b1-a35c-4e73-9874-8c48df6540c2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:24.742741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddf2b1-a35c-4e73-9874-8c48df6540c2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:24.781920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251a49c-461f-4609-8f4d-68edd2540df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:24.781920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a49c-461f-4609-8f4d-68edd2540df3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:24.781920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a49c-461f-4609-8f4d-68edd2540df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:24.781920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251a49c-461f-4609-8f4d-68edd2540df3 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:24.782585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251a49c-461f-4609-8f4d-68edd2540df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:24.782585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251a49c-461f-4609-8f4d-68edd2540df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:24.783192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251a49c-461f-4609-8f4d-68edd2540df3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:24.800040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 342/694] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:24.810351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251a49c-461f-4609-8f4d-68edd2540df3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:24.813653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2e490-7efb-4158-9d0c-f5c750fba70f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:24.887710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:24.902634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709d0dc-09e0-4084-80e2-3713680df705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:24.903336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709d0dc-09e0-4084-80e2-3713680df705 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:24.903626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709d0dc-09e0-4084-80e2-3713680df705 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:24.903907 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709d0dc-09e0-4084-80e2-3713680df705 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:24.904774 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709d0dc-09e0-4084-80e2-3713680df705 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:24.905090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709d0dc-09e0-4084-80e2-3713680df705 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:24.906580 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709d0dc-09e0-4084-80e2-3713680df705 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:24.929228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 353/695] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 4 09:02:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:24.935297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709d0dc-09e0-4084-80e2-3713680df705 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:24.939963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251a49c-461f-4609-8f4d-68edd2540df3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:24.978269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc3ff1-fd39-4f1e-8a18-692edfd7953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:24.978269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc3ff1-fd39-4f1e-8a18-692edfd7953f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:24.978269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc3ff1-fd39-4f1e-8a18-692edfd7953f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:24.978269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc3ff1-fd39-4f1e-8a18-692edfd7953f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:24.985494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc3ff1-fd39-4f1e-8a18-692edfd7953f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:24.987212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc3ff1-fd39-4f1e-8a18-692edfd7953f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d4fc3ff1-fd39-4f1e-8a18-692edfd7953f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:25.009215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:25.010169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:25.011096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:25.011096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:25.011096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:25.011096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:25.011096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:25.011096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:25.011096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:25.011096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:25.011096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 343/696] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:25.019388 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc3ff1-fd39-4f1e-8a18-692edfd7953f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:25.053755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:25.129423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709d0dc-09e0-4084-80e2-3713680df705 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:25.173108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc3ff1-fd39-4f1e-8a18-692edfd7953f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:25.210321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:25.214500 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90651925-98f4-431f-b280-c12f5cbe2419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:25.215197 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59ec05-100f-491c-a064-de87183cc53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:25.216786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59ec05-100f-491c-a064-de87183cc53f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:25.216946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90651925-98f4-431f-b280-c12f5cbe2419 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:25.217620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59ec05-100f-491c-a064-de87183cc53f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:25.218434 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59ec05-100f-491c-a064-de87183cc53f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:25.218568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90651925-98f4-431f-b280-c12f5cbe2419 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:25.219601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90651925-98f4-431f-b280-c12f5cbe2419 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:25.220547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90651925-98f4-431f-b280-c12f5cbe2419 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:25.221371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90651925-98f4-431f-b280-c12f5cbe2419 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:25.226126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90651925-98f4-431f-b280-c12f5cbe2419 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:25.249946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 344/697] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:25.251706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90651925-98f4-431f-b280-c12f5cbe2419 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:25.337939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59ec05-100f-491c-a064-de87183cc53f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:25.339410 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59ec05-100f-491c-a064-de87183cc53f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:25.400714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90651925-98f4-431f-b280-c12f5cbe2419 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:25.415617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:25.445928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa352aa5-c5e4-43dc-be11-0b98bb03c880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:25.446870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa352aa5-c5e4-43dc-be11-0b98bb03c880 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:25.447228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa352aa5-c5e4-43dc-be11-0b98bb03c880 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:25.447577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa352aa5-c5e4-43dc-be11-0b98bb03c880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:25.448745 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa352aa5-c5e4-43dc-be11-0b98bb03c880 None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=8cbb65dbfadf4c63a272f5318efa875a) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:25.449133 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa352aa5-c5e4-43dc-be11-0b98bb03c880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:25.451431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa352aa5-c5e4-43dc-be11-0b98bb03c880 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:25.459693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 345/698] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:25 2022] GET /identity/v3/role_assignments?user.id=8cbb65dbfadf4c63a272f5318efa875a&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:25.461717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a3cb6-ce2a-4abd-b137-0dce8007fed5 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:25.462219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a3cb6-ce2a-4abd-b137-0dce8007fed5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:25.462573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a3cb6-ce2a-4abd-b137-0dce8007fed5 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:25.465004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 346/699] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:25.474224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650170f4-542a-4cb2-8e8c-372516f4db32 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:25.474543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650170f4-542a-4cb2-8e8c-372516f4db32 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:25.474927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650170f4-542a-4cb2-8e8c-372516f4db32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:25.475907 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 354/700] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 04 09:02:25.484344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59ec05-100f-491c-a064-de87183cc53f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:25.503382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-650170f4-542a-4cb2-8e8c-372516f4db32 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:25.504778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-650170f4-542a-4cb2-8e8c-372516f4db32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:25.609760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:25.629624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59ec05-100f-491c-a064-de87183cc53f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:25.659120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 347/701] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:25.678221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4da5cb-7d53-4183-82f4-03c6228b7629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:25.679047 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4da5cb-7d53-4183-82f4-03c6228b7629 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:25.679332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4da5cb-7d53-4183-82f4-03c6228b7629 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:25.679606 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4da5cb-7d53-4183-82f4-03c6228b7629 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:25.681761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650170f4-542a-4cb2-8e8c-372516f4db32 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:25.697278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4da5cb-7d53-4183-82f4-03c6228b7629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:25.698492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4da5cb-7d53-4183-82f4-03c6228b7629 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-dd4da5cb-7d53-4183-82f4-03c6228b7629 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:25.718049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:25.718994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:25.719891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:25.719891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:25.719891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:25.719891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:25.719891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 355/702] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:25.730247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4da5cb-7d53-4183-82f4-03c6228b7629 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:25.748634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:25.823746 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650170f4-542a-4cb2-8e8c-372516f4db32 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:25.865249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e37678-0b55-4aa0-a893-d9af2e388d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:25.866087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e37678-0b55-4aa0-a893-d9af2e388d78 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:25.866420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e37678-0b55-4aa0-a893-d9af2e388d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:25.866701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e37678-0b55-4aa0-a893-d9af2e388d78 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:25.867367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e37678-0b55-4aa0-a893-d9af2e388d78 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:25.868795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e37678-0b55-4aa0-a893-d9af2e388d78 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:25.869376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4da5cb-7d53-4183-82f4-03c6228b7629 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-20e37678-0b55-4aa0-a893-d9af2e388d78 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:25.883985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:25.885270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:02:25.886342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:25.887738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 348/703] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jul 4 09:02:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:25.890312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70491d-1a39-45e7-94f9-6939d74e04cd None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:25.890596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70491d-1a39-45e7-94f9-6939d74e04cd None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:25.890873 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70491d-1a39-45e7-94f9-6939d74e04cd None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:25.892481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 349/704] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:25.895216 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70491d-1a39-45e7-94f9-6939d74e04cd None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:25.909685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e83f07-26bd-407e-8952-37d961038051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:25.910890 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e83f07-26bd-407e-8952-37d961038051 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:25.911325 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e83f07-26bd-407e-8952-37d961038051 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:25.911751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e83f07-26bd-407e-8952-37d961038051 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:25.912650 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e83f07-26bd-407e-8952-37d961038051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:25.913214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e83f07-26bd-407e-8952-37d961038051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:25.914890 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e83f07-26bd-407e-8952-37d961038051 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:25.927992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 356/705] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:25.929482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9834c5-d5bc-46a2-90bb-fc37a86b5f1c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:25.930010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9834c5-d5bc-46a2-90bb-fc37a86b5f1c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:25.930438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9834c5-d5bc-46a2-90bb-fc37a86b5f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:25.940764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:25.951315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1f9834c5-d5bc-46a2-90bb-fc37a86b5f1c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:25.952873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1f9834c5-d5bc-46a2-90bb-fc37a86b5f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:26.029938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70491d-1a39-45e7-94f9-6939d74e04cd None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:26.031595 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:26.042034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 357/706] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:26.042926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9834c5-d5bc-46a2-90bb-fc37a86b5f1c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:26.068760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ff2c6-6b70-45e6-bd32-813d46f9891a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:26.069677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ff2c6-6b70-45e6-bd32-813d46f9891a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:26.070194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ff2c6-6b70-45e6-bd32-813d46f9891a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:26.070649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ff2c6-6b70-45e6-bd32-813d46f9891a None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:26.071510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ff2c6-6b70-45e6-bd32-813d46f9891a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:26.071961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ff2c6-6b70-45e6-bd32-813d46f9891a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:26.073489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ff2c6-6b70-45e6-bd32-813d46f9891a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:26.078946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 350/707] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 4 09:02:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:26.080892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ff2c6-6b70-45e6-bd32-813d46f9891a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:26.161379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9834c5-d5bc-46a2-90bb-fc37a86b5f1c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:26.188741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:26.194048 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ff2c6-6b70-45e6-bd32-813d46f9891a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:26.195929 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729a245-b48c-4243-9a98-23e16da79f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:26.196705 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729a245-b48c-4243-9a98-23e16da79f08 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:26.197054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729a245-b48c-4243-9a98-23e16da79f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:26.198084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729a245-b48c-4243-9a98-23e16da79f08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:26.204745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729a245-b48c-4243-9a98-23e16da79f08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:26.206317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729a245-b48c-4243-9a98-23e16da79f08 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f729a245-b48c-4243-9a98-23e16da79f08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:26.220688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:26.221626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:26.222654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:26.222654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:26.222654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:26.222654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:26.222654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:26.222654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:26.222654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:26.222654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:26.222654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 358/708] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:26.223223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729a245-b48c-4243-9a98-23e16da79f08 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:26.244326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e82557-a47d-45e3-86dc-6e7fac55a865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:26.245180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e82557-a47d-45e3-86dc-6e7fac55a865 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:26.245562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e82557-a47d-45e3-86dc-6e7fac55a865 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:26.245843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e82557-a47d-45e3-86dc-6e7fac55a865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:26.258352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e82557-a47d-45e3-86dc-6e7fac55a865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:26.259922 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e82557-a47d-45e3-86dc-6e7fac55a865 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c3e82557-a47d-45e3-86dc-6e7fac55a865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:26.280650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:26.281613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:26.282573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:26.282573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:26.282573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:26.282573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:26.282573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 351/709] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:26.283297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e82557-a47d-45e3-86dc-6e7fac55a865 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:26.287512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:26.329559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729a245-b48c-4243-9a98-23e16da79f08 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:26.362999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7a550-bfd3-41af-a6cf-092febb77173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:26.363717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7a550-bfd3-41af-a6cf-092febb77173 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:26.363993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7a550-bfd3-41af-a6cf-092febb77173 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:26.364266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7a550-bfd3-41af-a6cf-092febb77173 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:26.365549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7a550-bfd3-41af-a6cf-092febb77173 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:26.365549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7a550-bfd3-41af-a6cf-092febb77173 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:26.366690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7a550-bfd3-41af-a6cf-092febb77173 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:26.372622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 359/710] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 4 09:02:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:26.373553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7a550-bfd3-41af-a6cf-092febb77173 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:26.400438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e82557-a47d-45e3-86dc-6e7fac55a865 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:26.458343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d11a1-5e7a-460f-bbe3-a5bb9422be2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:26.459912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d11a1-5e7a-460f-bbe3-a5bb9422be2d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:26.460637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d11a1-5e7a-460f-bbe3-a5bb9422be2d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:26.461153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d11a1-5e7a-460f-bbe3-a5bb9422be2d None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:26.462946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d11a1-5e7a-460f-bbe3-a5bb9422be2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:26.463828 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d11a1-5e7a-460f-bbe3-a5bb9422be2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:26.466366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d11a1-5e7a-460f-bbe3-a5bb9422be2d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:26.485452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 352/711] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:26.487208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be8589-ca3d-4b6e-8559-9ca90dda07fe None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:26.487638 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be8589-ca3d-4b6e-8559-9ca90dda07fe None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:26.488254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be8589-ca3d-4b6e-8559-9ca90dda07fe None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:26.490750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 353/712] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:26.491871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be8589-ca3d-4b6e-8559-9ca90dda07fe None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:26.508644 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:26.527784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7a550-bfd3-41af-a6cf-092febb77173 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:26.556887 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:26.571215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68fccd-585b-4cd7-ba2e-b675f67b02f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:26.571215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68fccd-585b-4cd7-ba2e-b675f67b02f6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:26.571215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68fccd-585b-4cd7-ba2e-b675f67b02f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:26.571611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68fccd-585b-4cd7-ba2e-b675f67b02f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:26.572072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68fccd-585b-4cd7-ba2e-b675f67b02f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:26.572380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68fccd-585b-4cd7-ba2e-b675f67b02f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:26.573763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68fccd-585b-4cd7-ba2e-b675f67b02f6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:26.588116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 360/713] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:26.590563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39428b-c7a2-431e-ae0e-fb3ade8ab0ae None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:26.590563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39428b-c7a2-431e-ae0e-fb3ade8ab0ae None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:26.590563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39428b-c7a2-431e-ae0e-fb3ade8ab0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:26.613572 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6c39428b-c7a2-431e-ae0e-fb3ade8ab0ae None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:26.615311 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6c39428b-c7a2-431e-ae0e-fb3ade8ab0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:26.615802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be8589-ca3d-4b6e-8559-9ca90dda07fe None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:26.653202 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3676-feae-4970-bb22-8fb224208c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:26.654066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3676-feae-4970-bb22-8fb224208c06 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:26.654665 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3676-feae-4970-bb22-8fb224208c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:26.654801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3676-feae-4970-bb22-8fb224208c06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:26.655716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e3676-feae-4970-bb22-8fb224208c06 None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=739f3b5eaef542c29a0cf0b475c0d965) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:26.656116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e3676-feae-4970-bb22-8fb224208c06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:26.657990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e3676-feae-4970-bb22-8fb224208c06 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:26.668708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 354/714] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:26 2022] GET /identity/v3/role_assignments?user.id=739f3b5eaef542c29a0cf0b475c0d965&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=2302517080ac42578aa81c606dd0ff2f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:26.669827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3676-feae-4970-bb22-8fb224208c06 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:26.751448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 361/715] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:26.766163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:26.801605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39428b-c7a2-431e-ae0e-fb3ade8ab0ae None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:26.802159 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3676-feae-4970-bb22-8fb224208c06 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:26.846115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70fca8-11ac-40a5-b0b5-eb1ad9064e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:26.847244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70fca8-11ac-40a5-b0b5-eb1ad9064e5b None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:26.848251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70fca8-11ac-40a5-b0b5-eb1ad9064e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:26.848706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70fca8-11ac-40a5-b0b5-eb1ad9064e5b None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:26.849591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70fca8-11ac-40a5-b0b5-eb1ad9064e5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:26.851583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70fca8-11ac-40a5-b0b5-eb1ad9064e5b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:26.902056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 355/716] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:02:26 2022] POST /identity/v3/users => generated 254 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:26.947922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39428b-c7a2-431e-ae0e-fb3ade8ab0ae None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:26.966062 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:26.990771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dee0d3-6c5a-4392-9759-a86fe8526897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:26.991695 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee0d3-6c5a-4392-9759-a86fe8526897 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:26.992023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee0d3-6c5a-4392-9759-a86fe8526897 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:26.992394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee0d3-6c5a-4392-9759-a86fe8526897 None admin] PATH_INFO: `/v3/services/network` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:26.993071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dee0d3-6c5a-4392-9759-a86fe8526897 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:26.994551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dee0d3-6c5a-4392-9759-a86fe8526897 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-31dee0d3-6c5a-4392-9759-a86fe8526897 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:27.012898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:27.014006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:27.015296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 362/717] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:26 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:27.023035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70fca8-11ac-40a5-b0b5-eb1ad9064e5b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:27.202851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70fca8-11ac-40a5-b0b5-eb1ad9064e5b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:27.235791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:27.245087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7e74a-0b51-4f7c-9648-1137f0d672f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:27.246944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e74a-0b51-4f7c-9648-1137f0d672f7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:27.247440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e74a-0b51-4f7c-9648-1137f0d672f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:27.247940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e74a-0b51-4f7c-9648-1137f0d672f7 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:27.248865 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7e74a-0b51-4f7c-9648-1137f0d672f7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:27.249369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7e74a-0b51-4f7c-9648-1137f0d672f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:27.251015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7e74a-0b51-4f7c-9648-1137f0d672f7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:27.261947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dee0d3-6c5a-4392-9759-a86fe8526897 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:27.263439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 356/718] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:02:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:27.395098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dee0d3-6c5a-4392-9759-a86fe8526897 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:27.435903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7934367-9a40-42d0-abdf-c3c93ac68a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:27.436575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7934367-9a40-42d0-abdf-c3c93ac68a17 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:27.436861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7934367-9a40-42d0-abdf-c3c93ac68a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:27.437137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7934367-9a40-42d0-abdf-c3c93ac68a17 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:27.437856 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7934367-9a40-42d0-abdf-c3c93ac68a17 None admin] RBAC: Adding query filter params (type=network) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:27.438200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7934367-9a40-42d0-abdf-c3c93ac68a17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:27.439595 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7934367-9a40-42d0-abdf-c3c93ac68a17 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:27.444787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 363/719] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:02:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:27.955679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704eafc-40b9-49f4-aafb-81019dbc0048 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:27.956087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704eafc-40b9-49f4-aafb-81019dbc0048 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:27.956260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704eafc-40b9-49f4-aafb-81019dbc0048 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:27.958172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 357/720] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:27.963167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2736ea-5cd6-4cfd-b8a8-5f37e001dd3b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:27.963640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2736ea-5cd6-4cfd-b8a8-5f37e001dd3b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:27.963747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2736ea-5cd6-4cfd-b8a8-5f37e001dd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:27.987056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1d2736ea-5cd6-4cfd-b8a8-5f37e001dd3b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:27.988368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d2736ea-5cd6-4cfd-b8a8-5f37e001dd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:28.032282 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:28.076511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 364/721] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:28.135504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704eafc-40b9-49f4-aafb-81019dbc0048 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:28.289485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704eafc-40b9-49f4-aafb-81019dbc0048 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:28.341983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ef278-def2-46a2-9cf7-711b2c394991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:28.343149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ef278-def2-46a2-9cf7-711b2c394991 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:28.343648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ef278-def2-46a2-9cf7-711b2c394991 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:28.344108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ef278-def2-46a2-9cf7-711b2c394991 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:28.360621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ef278-def2-46a2-9cf7-711b2c394991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-3d0ef278-def2-46a2-9cf7-711b2c394991 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:28.368827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:28.369766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:28.370655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:28.373354 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 358/722] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:28.379433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2736ea-5cd6-4cfd-b8a8-5f37e001dd3b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:28.500964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2736ea-5cd6-4cfd-b8a8-5f37e001dd3b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:28.537020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e90eb-ebe8-4bf7-ac18-743a19be489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:28.537694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e90eb-ebe8-4bf7-ac18-743a19be489d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:28.538009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e90eb-ebe8-4bf7-ac18-743a19be489d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:28.538286 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e90eb-ebe8-4bf7-ac18-743a19be489d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:28.539042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e90eb-ebe8-4bf7-ac18-743a19be489d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:28.539357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e90eb-ebe8-4bf7-ac18-743a19be489d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:28.540807 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e90eb-ebe8-4bf7-ac18-743a19be489d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:28.549383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 365/723] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:28.555705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ef278-def2-46a2-9cf7-711b2c394991 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:28.686282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:28.748714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ef278-def2-46a2-9cf7-711b2c394991 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:28.797499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32361672-2e0e-44ce-9db6-9bb5a247124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:28.798243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32361672-2e0e-44ce-9db6-9bb5a247124c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:28.798529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32361672-2e0e-44ce-9db6-9bb5a247124c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:28.798806 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32361672-2e0e-44ce-9db6-9bb5a247124c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:28.811536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db8669-8ea8-4da5-9a6a-655e48033ac8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:28.811823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db8669-8ea8-4da5-9a6a-655e48033ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:28.812109 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db8669-8ea8-4da5-9a6a-655e48033ac8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:28.812849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32361672-2e0e-44ce-9db6-9bb5a247124c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:28.813764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 366/724] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:28.814344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32361672-2e0e-44ce-9db6-9bb5a247124c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:28.819448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e595634-5fe5-4691-be69-3bec1d95ec86 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:28.820010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e595634-5fe5-4691-be69-3bec1d95ec86 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:28.820560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e595634-5fe5-4691-be69-3bec1d95ec86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-32361672-2e0e-44ce-9db6-9bb5a247124c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:28.838526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:28.839557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:28.840530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:28.840530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:28.840530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:28.840530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:28.842551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 359/725] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:28.848278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32361672-2e0e-44ce-9db6-9bb5a247124c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:28.870073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9e595634-5fe5-4691-be69-3bec1d95ec86 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:28.871798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9e595634-5fe5-4691-be69-3bec1d95ec86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:28.942467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:28.986298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 367/726] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:02:29.012870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e595634-5fe5-4691-be69-3bec1d95ec86 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:29.020933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32361672-2e0e-44ce-9db6-9bb5a247124c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:29.028452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:29.086851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe0b2f-8920-403e-ba87-750e5f76002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:29.087867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe0b2f-8920-403e-ba87-750e5f76002c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:29.088953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe0b2f-8920-403e-ba87-750e5f76002c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:29.089423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe0b2f-8920-403e-ba87-750e5f76002c None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:29.091392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe0b2f-8920-403e-ba87-750e5f76002c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:29.091884 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe0b2f-8920-403e-ba87-750e5f76002c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:29.093619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe0b2f-8920-403e-ba87-750e5f76002c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:29.125444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 360/727] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:29.128371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe0b2f-8920-403e-ba87-750e5f76002c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:29.186138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e595634-5fe5-4691-be69-3bec1d95ec86 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:29.229533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116938d-42ad-438d-83ba-b79d8e083958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:29.230322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116938d-42ad-438d-83ba-b79d8e083958 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:29.230614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116938d-42ad-438d-83ba-b79d8e083958 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:29.230897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116938d-42ad-438d-83ba-b79d8e083958 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:29.231758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116938d-42ad-438d-83ba-b79d8e083958 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:29.233292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116938d-42ad-438d-83ba-b79d8e083958 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:29.253418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 368/728] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:02:29 2022] POST /identity/v3/services => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:02:29.255310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c850-81d5-4108-9fb8-edc68f23edbf None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:29.255653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c850-81d5-4108-9fb8-edc68f23edbf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:29.255981 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c850-81d5-4108-9fb8-edc68f23edbf None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:29.257915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 369/729] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:29.263489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25202de-54d5-4500-ab1e-474a7346b531 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:29.263911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25202de-54d5-4500-ab1e-474a7346b531 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:29.264266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25202de-54d5-4500-ab1e-474a7346b531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:29.283504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f25202de-54d5-4500-ab1e-474a7346b531 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:29.283714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe0b2f-8920-403e-ba87-750e5f76002c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:29.284916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f25202de-54d5-4500-ab1e-474a7346b531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:29.306414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:29.335130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb6b44-3614-43a0-a2bc-bac1a89d5192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:29.336175 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb6b44-3614-43a0-a2bc-bac1a89d5192 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:29.336338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb6b44-3614-43a0-a2bc-bac1a89d5192 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:29.336707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb6b44-3614-43a0-a2bc-bac1a89d5192 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:29.345018 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb6b44-3614-43a0-a2bc-bac1a89d5192 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:29.346573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb6b44-3614-43a0-a2bc-bac1a89d5192 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-17cb6b44-3614-43a0-a2bc-bac1a89d5192 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:29.361744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:29.362814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:29.363854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:29.363854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:29.363854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:29.363854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:29.363854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:29.363854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:29.363854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:29.363854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:29.363854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:29.363854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:29.363854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 361/730] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:29.372208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 370/731] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 04 09:02:29.372967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb6b44-3614-43a0-a2bc-bac1a89d5192 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:29.398340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25202de-54d5-4500-ab1e-474a7346b531 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:29.431470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:29.530581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb6b44-3614-43a0-a2bc-bac1a89d5192 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:29.547880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25202de-54d5-4500-ab1e-474a7346b531 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:29.570316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eff85b-6548-4c71-8de5-09c795b8693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:29.571109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eff85b-6548-4c71-8de5-09c795b8693e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:29.571463 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eff85b-6548-4c71-8de5-09c795b8693e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:29.571805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eff85b-6548-4c71-8de5-09c795b8693e None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:29.572738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eff85b-6548-4c71-8de5-09c795b8693e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:29.573113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eff85b-6548-4c71-8de5-09c795b8693e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:29.574653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eff85b-6548-4c71-8de5-09c795b8693e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:29.593716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04c48a-ffff-485a-97b3-a03ba7696f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:29.594828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04c48a-ffff-485a-97b3-a03ba7696f0e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:29.595166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04c48a-ffff-485a-97b3-a03ba7696f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:29.595386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04c48a-ffff-485a-97b3-a03ba7696f0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:29.596309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 362/732] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 4 09:02:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:29.597750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebb9d0-abb3-4533-9c6d-f7e3920ed011 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:29.598171 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebb9d0-abb3-4533-9c6d-f7e3920ed011 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:29.598466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebb9d0-abb3-4533-9c6d-f7e3920ed011 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:29.600383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 363/733] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:29.601923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebb9d0-abb3-4533-9c6d-f7e3920ed011 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:29.608312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04c48a-ffff-485a-97b3-a03ba7696f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ae04c48a-ffff-485a-97b3-a03ba7696f0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:29.620586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:29.621677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:29.622592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 371/734] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:29.623819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e792b-6a43-4023-b1dc-6e13a399d463 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:29.623976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e792b-6a43-4023-b1dc-6e13a399d463 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:29.624234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e792b-6a43-4023-b1dc-6e13a399d463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:29.647866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cb1e792b-6a43-4023-b1dc-6e13a399d463 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:29.649118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cb1e792b-6a43-4023-b1dc-6e13a399d463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:29.662471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:29.698500 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:29.729465 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebb9d0-abb3-4533-9c6d-f7e3920ed011 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:29.731580 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 372/735] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:29.738085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e792b-6a43-4023-b1dc-6e13a399d463 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:29.785048 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19856b5-198f-4d64-9335-a50d7f0555ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:29.786279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19856b5-198f-4d64-9335-a50d7f0555ab None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:29.786700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19856b5-198f-4d64-9335-a50d7f0555ab None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:29.787201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19856b5-198f-4d64-9335-a50d7f0555ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:29.861717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19856b5-198f-4d64-9335-a50d7f0555ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:29.863682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19856b5-198f-4d64-9335-a50d7f0555ab None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:29.871886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e792b-6a43-4023-b1dc-6e13a399d463 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:29.922540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf9edf-0763-45be-9a47-c004993ce81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:29.923249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9edf-0763-45be-9a47-c004993ce81a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:29.923534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9edf-0763-45be-9a47-c004993ce81a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:29.923812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9edf-0763-45be-9a47-c004993ce81a None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:29.925103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf9edf-0763-45be-9a47-c004993ce81a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:29.925424 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf9edf-0763-45be-9a47-c004993ce81a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:29.927188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf9edf-0763-45be-9a47-c004993ce81a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:29.939000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 373/736] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:29.940423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf9edf-0763-45be-9a47-c004993ce81a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:29.954239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:29.957791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:29.982452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 364/737] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:29 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 04 09:02:29.983784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19856b5-198f-4d64-9335-a50d7f0555ab None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:30.074838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf9edf-0763-45be-9a47-c004993ce81a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:30.088548 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19856b5-198f-4d64-9335-a50d7f0555ab None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:30.111435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa87e6f-638c-45b0-af22-cc5b5ff0fc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:30.112428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa87e6f-638c-45b0-af22-cc5b5ff0fc82 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.112654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa87e6f-638c-45b0-af22-cc5b5ff0fc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.112979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa87e6f-638c-45b0-af22-cc5b5ff0fc82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.124164 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa87e6f-638c-45b0-af22-cc5b5ff0fc82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:30.130862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed9d14-9275-4eeb-a170-d2ef8a43f193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:30.131737 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed9d14-9275-4eeb-a170-d2ef8a43f193 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.132026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed9d14-9275-4eeb-a170-d2ef8a43f193 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.132298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed9d14-9275-4eeb-a170-d2ef8a43f193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-eaa87e6f-638c-45b0-af22-cc5b5ff0fc82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:30.132482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:30.133347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:30.134232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:30.138078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 374/738] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:30.138934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa87e6f-638c-45b0-af22-cc5b5ff0fc82 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:30.146639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed9d14-9275-4eeb-a170-d2ef8a43f193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:30.148112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed9d14-9275-4eeb-a170-d2ef8a43f193 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-e0ed9d14-9275-4eeb-a170-d2ef8a43f193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:30.168827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:30.169744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:30.170697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:30.170697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:30.170697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:30.170697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:30.170697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 365/739] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:30.171968 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edbebd-ffe8-47ff-abd3-46d1469d692e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.172258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edbebd-ffe8-47ff-abd3-46d1469d692e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.172538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edbebd-ffe8-47ff-abd3-46d1469d692e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.173978 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 366/740] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:30.174711 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edbebd-ffe8-47ff-abd3-46d1469d692e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:30.209478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:30.216172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:30.250529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa87e6f-638c-45b0-af22-cc5b5ff0fc82 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:30.284792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edbebd-ffe8-47ff-abd3-46d1469d692e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:30.285427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f8eb9-2f39-4c34-bb1b-602eeec744cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:30.286268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f8eb9-2f39-4c34-bb1b-602eeec744cf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.286555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f8eb9-2f39-4c34-bb1b-602eeec744cf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.286834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f8eb9-2f39-4c34-bb1b-602eeec744cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.298197 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f8eb9-2f39-4c34-bb1b-602eeec744cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:30.299653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f8eb9-2f39-4c34-bb1b-602eeec744cf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9b4f8eb9-2f39-4c34-bb1b-602eeec744cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:30.316710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:30.317665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:30.318634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:30.318634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:30.318634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:30.318634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:30.318634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 375/741] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:30.318915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f8eb9-2f39-4c34-bb1b-602eeec744cf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:30.328012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea39d3a-9e04-47e0-ac7e-a8d18d60a03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:30.329080 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39d3a-9e04-47e0-ac7e-a8d18d60a03d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.329620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39d3a-9e04-47e0-ac7e-a8d18d60a03d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.330140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39d3a-9e04-47e0-ac7e-a8d18d60a03d None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.331256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea39d3a-9e04-47e0-ac7e-a8d18d60a03d None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:30.332116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea39d3a-9e04-47e0-ac7e-a8d18d60a03d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:30.333557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea39d3a-9e04-47e0-ac7e-a8d18d60a03d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:30.342549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 367/742] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:30.343584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b8d7a-fa15-4af7-8893-00e085444730 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.343871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b8d7a-fa15-4af7-8893-00e085444730 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.344146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b8d7a-fa15-4af7-8893-00e085444730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.361013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-579b8d7a-fa15-4af7-8893-00e085444730 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:30.362273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-579b8d7a-fa15-4af7-8893-00e085444730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:30.428082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 368/743] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 04 09:02:30.428846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b8d7a-fa15-4af7-8893-00e085444730 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:30.439035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f8eb9-2f39-4c34-bb1b-602eeec744cf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:30.459424 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:30.464188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:30.479433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e41a3c-8937-4249-afbe-1f87e7f49be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:30.480202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e41a3c-8937-4249-afbe-1f87e7f49be5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.480557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e41a3c-8937-4249-afbe-1f87e7f49be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.480906 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e41a3c-8937-4249-afbe-1f87e7f49be5 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.481707 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e41a3c-8937-4249-afbe-1f87e7f49be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:30.482076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e41a3c-8937-4249-afbe-1f87e7f49be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:30.483454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e41a3c-8937-4249-afbe-1f87e7f49be5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:30.499198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 376/744] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:30.500339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e41a3c-8937-4249-afbe-1f87e7f49be5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:30.551235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b8d7a-fa15-4af7-8893-00e085444730 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:30.587689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d49d63-320a-4c12-a28c-5cd19086486c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:30.588601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d49d63-320a-4c12-a28c-5cd19086486c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.588722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d49d63-320a-4c12-a28c-5cd19086486c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.588898 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d49d63-320a-4c12-a28c-5cd19086486c None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.589633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d49d63-320a-4c12-a28c-5cd19086486c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:30.589966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d49d63-320a-4c12-a28c-5cd19086486c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:30.591360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d49d63-320a-4c12-a28c-5cd19086486c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:30.603286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 369/745] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:30.604075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d49d63-320a-4c12-a28c-5cd19086486c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:30.618287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e41a3c-8937-4249-afbe-1f87e7f49be5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:30.660511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08888429-0a5e-4ff2-9418-621fb3b504cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:30.661831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08888429-0a5e-4ff2-9418-621fb3b504cf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.662500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08888429-0a5e-4ff2-9418-621fb3b504cf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.663327 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08888429-0a5e-4ff2-9418-621fb3b504cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.674055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08888429-0a5e-4ff2-9418-621fb3b504cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:30.675563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08888429-0a5e-4ff2-9418-621fb3b504cf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-08888429-0a5e-4ff2-9418-621fb3b504cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:30.694392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:30.695300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:30.696213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:30.696213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:30.696213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:30.696213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:30.696213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 377/746] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:30.697157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08888429-0a5e-4ff2-9418-621fb3b504cf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:30.697624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:30.718520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d49d63-320a-4c12-a28c-5cd19086486c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:30.733137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:30.755791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad577ac8-f821-46ed-b245-6cdc71406cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:30.756588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad577ac8-f821-46ed-b245-6cdc71406cd2 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.756868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad577ac8-f821-46ed-b245-6cdc71406cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.757147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad577ac8-f821-46ed-b245-6cdc71406cd2 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.757892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad577ac8-f821-46ed-b245-6cdc71406cd2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:30.759327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad577ac8-f821-46ed-b245-6cdc71406cd2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:30.785911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 370/747] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:02:30 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:30.785911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad577ac8-f821-46ed-b245-6cdc71406cd2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:30.844361 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08888429-0a5e-4ff2-9418-621fb3b504cf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:30.903578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0ab3c-8877-4909-a6b9-2061048ec1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:30.904590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0ab3c-8877-4909-a6b9-2061048ec1eb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.904763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad577ac8-f821-46ed-b245-6cdc71406cd2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:30.904987 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0ab3c-8877-4909-a6b9-2061048ec1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.905320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0ab3c-8877-4909-a6b9-2061048ec1eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.916244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0ab3c-8877-4909-a6b9-2061048ec1eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:30.918436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0ab3c-8877-4909-a6b9-2061048ec1eb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-03d0ab3c-8877-4909-a6b9-2061048ec1eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:30.939773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:30.940788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:30.941830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:30.941830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:30.941830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:30.941830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:30.941830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:30.941830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:30.941830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:30.941830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:30.941830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:30.941830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:30.941830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 378/748] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 4 09:02:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:30.942531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0ab3c-8877-4909-a6b9-2061048ec1eb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:30.950278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7e84d-fb5c-405c-ac34-be2c63e62368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:30.951021 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e84d-fb5c-405c-ac34-be2c63e62368 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:30.951303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e84d-fb5c-405c-ac34-be2c63e62368 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:30.951635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e84d-fb5c-405c-ac34-be2c63e62368 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:30.962068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7e84d-fb5c-405c-ac34-be2c63e62368 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:30.963633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7e84d-fb5c-405c-ac34-be2c63e62368 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:30.966603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-38c7e84d-fb5c-405c-ac34-be2c63e62368 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:30.980738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:30.981678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:30.982649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:30.982649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:30.982649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:30.982649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:30.982649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:30.982649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:30.982649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:30.982649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:30.983546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 371/749] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:30.985413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7e84d-fb5c-405c-ac34-be2c63e62368 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:31.042984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:31.059513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0ab3c-8877-4909-a6b9-2061048ec1eb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:31.099616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789e80a-64c1-4b92-8051-4c0fefb95c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:31.101096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789e80a-64c1-4b92-8051-4c0fefb95c45 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.101586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789e80a-64c1-4b92-8051-4c0fefb95c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.102150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789e80a-64c1-4b92-8051-4c0fefb95c45 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.103127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789e80a-64c1-4b92-8051-4c0fefb95c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:31.103659 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789e80a-64c1-4b92-8051-4c0fefb95c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:31.105265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789e80a-64c1-4b92-8051-4c0fefb95c45 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:31.109129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7e84d-fb5c-405c-ac34-be2c63e62368 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:31.118782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 379/750] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:31.119957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789e80a-64c1-4b92-8051-4c0fefb95c45 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:31.146148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6eb9e-481d-4782-919d-d8ca93483649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:31.146825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6eb9e-481d-4782-919d-d8ca93483649 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.147110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6eb9e-481d-4782-919d-d8ca93483649 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.147386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6eb9e-481d-4782-919d-d8ca93483649 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.148239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6eb9e-481d-4782-919d-d8ca93483649 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:31.148547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6eb9e-481d-4782-919d-d8ca93483649 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:31.150019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6eb9e-481d-4782-919d-d8ca93483649 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:31.173461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 372/751] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 4 09:02:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:31.174417 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6eb9e-481d-4782-919d-d8ca93483649 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:31.230372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789e80a-64c1-4b92-8051-4c0fefb95c45 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:31.234830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:31.267082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e84197-6157-4d12-a993-ebf518501340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:31.268113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e84197-6157-4d12-a993-ebf518501340 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.268619 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e84197-6157-4d12-a993-ebf518501340 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.269149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e84197-6157-4d12-a993-ebf518501340 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.270439 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e84197-6157-4d12-a993-ebf518501340 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:31.270953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e84197-6157-4d12-a993-ebf518501340 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:31.272590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e84197-6157-4d12-a993-ebf518501340 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:31.279117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6eb9e-481d-4782-919d-d8ca93483649 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:31.286418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 380/752] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:31.287545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e84197-6157-4d12-a993-ebf518501340 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:31.296452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:31.314773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366c669-5a4b-475a-a100-4d4cd8a9b23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:31.315675 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366c669-5a4b-475a-a100-4d4cd8a9b23c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.315890 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366c669-5a4b-475a-a100-4d4cd8a9b23c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.316226 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366c669-5a4b-475a-a100-4d4cd8a9b23c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.324317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366c669-5a4b-475a-a100-4d4cd8a9b23c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:31.325783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366c669-5a4b-475a-a100-4d4cd8a9b23c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-7366c669-5a4b-475a-a100-4d4cd8a9b23c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:31.341614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:31.342678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:31.343684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:31.343684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:31.343684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:31.343684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:31.343684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:31.343684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:31.343684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:31.343684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:31.343684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:31.343684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:31.343684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 373/753] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 4 09:02:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:31.344285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366c669-5a4b-475a-a100-4d4cd8a9b23c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:31.405320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e84197-6157-4d12-a993-ebf518501340 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:31.440051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce15f0-cbc2-40f8-b41d-499e484472bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:31.440853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce15f0-cbc2-40f8-b41d-499e484472bb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.441212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce15f0-cbc2-40f8-b41d-499e484472bb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.441544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce15f0-cbc2-40f8-b41d-499e484472bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.452029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce15f0-cbc2-40f8-b41d-499e484472bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:31.453456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce15f0-cbc2-40f8-b41d-499e484472bb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:31.461120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366c669-5a4b-475a-a100-4d4cd8a9b23c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e9ce15f0-cbc2-40f8-b41d-499e484472bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:31.471463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:31.472466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:31.473393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:31.473393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:31.473393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:31.473393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:31.477285 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 381/754] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:31.478212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce15f0-cbc2-40f8-b41d-499e484472bb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:31.482712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:31.505618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716480b-c0a2-40e1-913e-91256c27ff41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:31.506312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716480b-c0a2-40e1-913e-91256c27ff41 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.506620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716480b-c0a2-40e1-913e-91256c27ff41 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.506867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716480b-c0a2-40e1-913e-91256c27ff41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.507707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716480b-c0a2-40e1-913e-91256c27ff41 None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=8cbb65dbfadf4c63a272f5318efa875a) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:31.508021 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716480b-c0a2-40e1-913e-91256c27ff41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:31.509464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716480b-c0a2-40e1-913e-91256c27ff41 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:31.515317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 374/755] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:31 2022] GET /identity/v3/role_assignments?user.id=8cbb65dbfadf4c63a272f5318efa875a&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=2302517080ac42578aa81c606dd0ff2f => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:31.516493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b78779-6ac4-4a23-9d0b-e57afd7d5c0c None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.516853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b78779-6ac4-4a23-9d0b-e57afd7d5c0c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.517203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b78779-6ac4-4a23-9d0b-e57afd7d5c0c None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.526159 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 375/756] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:31.526159 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b78779-6ac4-4a23-9d0b-e57afd7d5c0c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:31.548053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:31.595452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce15f0-cbc2-40f8-b41d-499e484472bb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:31.634268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870af9e-81ec-4349-ae83-453fdfcf0a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:31.636200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870af9e-81ec-4349-ae83-453fdfcf0a52 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.636647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870af9e-81ec-4349-ae83-453fdfcf0a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.636957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870af9e-81ec-4349-ae83-453fdfcf0a52 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.638056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870af9e-81ec-4349-ae83-453fdfcf0a52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:31.638469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870af9e-81ec-4349-ae83-453fdfcf0a52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:31.640097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870af9e-81ec-4349-ae83-453fdfcf0a52 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:31.649355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b78779-6ac4-4a23-9d0b-e57afd7d5c0c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:31.661108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 382/757] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 4 09:02:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:31.662717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095a5c2-a17d-4575-a994-f659f730d9fe None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.663102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095a5c2-a17d-4575-a994-f659f730d9fe None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.663499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095a5c2-a17d-4575-a994-f659f730d9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.681272 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1095a5c2-a17d-4575-a994-f659f730d9fe None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:31.682301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1095a5c2-a17d-4575-a994-f659f730d9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:31.691619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9baa06-2ae3-4a8d-b9b9-aa702671a970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:31.692681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9baa06-2ae3-4a8d-b9b9-aa702671a970 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.693008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9baa06-2ae3-4a8d-b9b9-aa702671a970 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.693305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9baa06-2ae3-4a8d-b9b9-aa702671a970 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.694268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9baa06-2ae3-4a8d-b9b9-aa702671a970 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:31.694794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9baa06-2ae3-4a8d-b9b9-aa702671a970 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:31.696417 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9baa06-2ae3-4a8d-b9b9-aa702671a970 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:31.709733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 376/758] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:31.710892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9baa06-2ae3-4a8d-b9b9-aa702671a970 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:31.752840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 383/759] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:31.758838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095a5c2-a17d-4575-a994-f659f730d9fe None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:31.761009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:31.819173 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9baa06-2ae3-4a8d-b9b9-aa702671a970 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:31.826253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:31.857866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f2480-13db-42f2-9679-77f3f53a6e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:31.858799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f2480-13db-42f2-9679-77f3f53a6e24 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.859475 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f2480-13db-42f2-9679-77f3f53a6e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.859986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f2480-13db-42f2-9679-77f3f53a6e24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.876112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095a5c2-a17d-4575-a994-f659f730d9fe None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:31.912176 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1568876-471d-49d1-b63f-ac60973b4ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:31.913186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1568876-471d-49d1-b63f-ac60973b4ead None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:31.913741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1568876-471d-49d1-b63f-ac60973b4ead None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:31.913905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1568876-471d-49d1-b63f-ac60973b4ead None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:31.920849 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1568876-471d-49d1-b63f-ac60973b4ead None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:31.922438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1568876-471d-49d1-b63f-ac60973b4ead None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f1568876-471d-49d1-b63f-ac60973b4ead None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:31.936461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:31.937586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:31.938749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:31.938749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:31.938749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:31.938749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:31.938749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:31.938749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:31.938749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:31.938749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:31.941447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 384/760] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:31.942574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1568876-471d-49d1-b63f-ac60973b4ead None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:31.945587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f2480-13db-42f2-9679-77f3f53a6e24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:31.948199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f2480-13db-42f2-9679-77f3f53a6e24 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:32.007349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:32.062025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1568876-471d-49d1-b63f-ac60973b4ead None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:32.080264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 377/761] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:02:32.080264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f2480-13db-42f2-9679-77f3f53a6e24 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:32.086316 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:32.119725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fd8ae-ed99-42ab-9e40-e362833b264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:32.121314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fd8ae-ed99-42ab-9e40-e362833b264b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:32.122237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fd8ae-ed99-42ab-9e40-e362833b264b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:32.122521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fd8ae-ed99-42ab-9e40-e362833b264b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:32.123192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fd8ae-ed99-42ab-9e40-e362833b264b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:32.126032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fd8ae-ed99-42ab-9e40-e362833b264b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cc3fd8ae-ed99-42ab-9e40-e362833b264b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:32.147209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:32.148207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:02:32.149100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:32.157955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 385/762] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jul 4 09:02:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:32.159103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fd8ae-ed99-42ab-9e40-e362833b264b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:32.198464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f2480-13db-42f2-9679-77f3f53a6e24 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:32.232302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166cc37-78fe-4f80-a33f-010fcf6ee3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:32.233095 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166cc37-78fe-4f80-a33f-010fcf6ee3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:32.233470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166cc37-78fe-4f80-a33f-010fcf6ee3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:32.234204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166cc37-78fe-4f80-a33f-010fcf6ee3c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:32.234976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166cc37-78fe-4f80-a33f-010fcf6ee3c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:32.235378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166cc37-78fe-4f80-a33f-010fcf6ee3c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:32.236937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166cc37-78fe-4f80-a33f-010fcf6ee3c1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:32.249185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 378/763] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:32.250145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166cc37-78fe-4f80-a33f-010fcf6ee3c1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:32.253128 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:32.275178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fd8ae-ed99-42ab-9e40-e362833b264b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:32.314534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199833ca-4f30-4366-9c97-a8a8a123056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:32.315640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199833ca-4f30-4366-9c97-a8a8a123056c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:32.316165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199833ca-4f30-4366-9c97-a8a8a123056c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:32.317379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199833ca-4f30-4366-9c97-a8a8a123056c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:32.329234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199833ca-4f30-4366-9c97-a8a8a123056c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:32.330955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199833ca-4f30-4366-9c97-a8a8a123056c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-199833ca-4f30-4366-9c97-a8a8a123056c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:32.351767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:32.352925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:32.354086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:32.354086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:32.354086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:32.354086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:32.360398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166cc37-78fe-4f80-a33f-010fcf6ee3c1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:32.361092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 386/764] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:32.362125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199833ca-4f30-4366-9c97-a8a8a123056c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:32.393116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128b78d-88fa-458b-a0cc-67e822f65b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:32.393962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128b78d-88fa-458b-a0cc-67e822f65b1d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:32.394504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128b78d-88fa-458b-a0cc-67e822f65b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:32.394803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128b78d-88fa-458b-a0cc-67e822f65b1d None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:32.395933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128b78d-88fa-458b-a0cc-67e822f65b1d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:32.396053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128b78d-88fa-458b-a0cc-67e822f65b1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:32.397512 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128b78d-88fa-458b-a0cc-67e822f65b1d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:32.406913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 379/765] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 4 09:02:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:32.407872 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128b78d-88fa-458b-a0cc-67e822f65b1d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:32.473390 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:32.479698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199833ca-4f30-4366-9c97-a8a8a123056c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:32.502636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:32.509578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128b78d-88fa-458b-a0cc-67e822f65b1d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:32.528798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed81f2-1a2c-4566-a5a8-50ff85eb5bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:32.528798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed81f2-1a2c-4566-a5a8-50ff85eb5bf1 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:32.528798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed81f2-1a2c-4566-a5a8-50ff85eb5bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:32.528798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed81f2-1a2c-4566-a5a8-50ff85eb5bf1 None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/739f3b5eaef542c29a0cf0b475c0d965/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:32.530714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed81f2-1a2c-4566-a5a8-50ff85eb5bf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=739f3b5eaef542c29a0cf0b475c0d965, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:32.532330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed81f2-1a2c-4566-a5a8-50ff85eb5bf1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:32.548021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 387/766] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:02:32 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/739f3b5eaef542c29a0cf0b475c0d965/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:02:32.548931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed394137-cacf-46e5-ac93-caea0df3fd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:32.549445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed81f2-1a2c-4566-a5a8-50ff85eb5bf1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:32.549818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed394137-cacf-46e5-ac93-caea0df3fd80 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:32.550205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed394137-cacf-46e5-ac93-caea0df3fd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:32.550756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed394137-cacf-46e5-ac93-caea0df3fd80 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:32.551658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed394137-cacf-46e5-ac93-caea0df3fd80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:32.552023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed394137-cacf-46e5-ac93-caea0df3fd80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:32.553808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed394137-cacf-46e5-ac93-caea0df3fd80 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:32.567746 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 380/767] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:32.575076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed394137-cacf-46e5-ac93-caea0df3fd80 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:32.674332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:32.724295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed81f2-1a2c-4566-a5a8-50ff85eb5bf1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:32.757807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed394137-cacf-46e5-ac93-caea0df3fd80 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:32.765845 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de26844-ff58-4bbf-89b1-837e1a2fc700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:32.766863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de26844-ff58-4bbf-89b1-837e1a2fc700 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:32.767341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de26844-ff58-4bbf-89b1-837e1a2fc700 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:32.767809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de26844-ff58-4bbf-89b1-837e1a2fc700 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:32.768716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de26844-ff58-4bbf-89b1-837e1a2fc700 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:32.769272 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de26844-ff58-4bbf-89b1-837e1a2fc700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:32.771719 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de26844-ff58-4bbf-89b1-837e1a2fc700 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:32.782287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 388/768] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 4 09:02:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:32.783428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de26844-ff58-4bbf-89b1-837e1a2fc700 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:32.788545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:32.799573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d762b5-f692-499b-b1a0-8ba86b26a27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:32.800379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d762b5-f692-499b-b1a0-8ba86b26a27f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:32.800691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d762b5-f692-499b-b1a0-8ba86b26a27f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:32.801034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d762b5-f692-499b-b1a0-8ba86b26a27f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:32.808882 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d762b5-f692-499b-b1a0-8ba86b26a27f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:32.810781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d762b5-f692-499b-b1a0-8ba86b26a27f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c3d762b5-f692-499b-b1a0-8ba86b26a27f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:32.825555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:32.826526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:32.827440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:32.827440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:32.827440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:32.827440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:32.827440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:32.827440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:32.827440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:32.827440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:32.827440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 381/769] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:32.833106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d762b5-f692-499b-b1a0-8ba86b26a27f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:32.927459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de26844-ff58-4bbf-89b1-837e1a2fc700 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:32.999242 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d7f88-a450-4d1b-bb9d-7d2ee7cb463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:32.999927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d7f88-a450-4d1b-bb9d-7d2ee7cb463d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:33.000213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d7f88-a450-4d1b-bb9d-7d2ee7cb463d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:33.000499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d7f88-a450-4d1b-bb9d-7d2ee7cb463d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:33.001307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d7f88-a450-4d1b-bb9d-7d2ee7cb463d None admin] RBAC: Adding query filter params (interface=public, service_id=842ecccbebdb47a7915044b0b3ef2039, region_id=RegionOne) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:33.001623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d7f88-a450-4d1b-bb9d-7d2ee7cb463d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:33.003382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d7f88-a450-4d1b-bb9d-7d2ee7cb463d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:33.017950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d762b5-f692-499b-b1a0-8ba86b26a27f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:33.024989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 389/770] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 4 09:02:32 2022] GET /identity/v3/endpoints?service_id=842ecccbebdb47a7915044b0b3ef2039&interface=public®ion_id=RegionOne => generated 194 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:33.030900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d7f88-a450-4d1b-bb9d-7d2ee7cb463d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:33.048984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:33.062642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c6f78-9c8d-4eea-9ed8-62a0f75e2c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:33.064094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c6f78-9c8d-4eea-9ed8-62a0f75e2c2f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:33.064843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c6f78-9c8d-4eea-9ed8-62a0f75e2c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:33.065324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c6f78-9c8d-4eea-9ed8-62a0f75e2c2f None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:33.066690 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c6f78-9c8d-4eea-9ed8-62a0f75e2c2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:33.067598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c6f78-9c8d-4eea-9ed8-62a0f75e2c2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:33.069818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c6f78-9c8d-4eea-9ed8-62a0f75e2c2f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:33.086142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 382/771] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:33.087777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83896b2-7720-4143-82e2-f1775d3406cb None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:33.088258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83896b2-7720-4143-82e2-f1775d3406cb None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:33.088723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83896b2-7720-4143-82e2-f1775d3406cb None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:33.090446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 383/772] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:33.091422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83896b2-7720-4143-82e2-f1775d3406cb None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:33.158622 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:33.174054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d7f88-a450-4d1b-bb9d-7d2ee7cb463d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:33.219264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d4261-0a9c-4042-a201-17819519e8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:33.220002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d4261-0a9c-4042-a201-17819519e8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:33.220293 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d4261-0a9c-4042-a201-17819519e8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:33.220581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d4261-0a9c-4042-a201-17819519e8c9 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:33.221540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d4261-0a9c-4042-a201-17819519e8c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:33.223218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d4261-0a9c-4042-a201-17819519e8c9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:33.238165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 390/773] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 4 09:02:33 2022] GET /identity/v3/services => generated 772 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:33.238165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb8b09-e010-4d70-865a-a9f016b489ae None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:33.238165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb8b09-e010-4d70-865a-a9f016b489ae None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:33.238165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb8b09-e010-4d70-865a-a9f016b489ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:33.238885 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83896b2-7720-4143-82e2-f1775d3406cb None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:33.271593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-dbbb8b09-e010-4d70-865a-a9f016b489ae None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:33.272846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dbbb8b09-e010-4d70-865a-a9f016b489ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:33.319630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc6fa3-e587-44e9-9bd1-220bfd59f759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:33.320235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc6fa3-e587-44e9-9bd1-220bfd59f759 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:33.320522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc6fa3-e587-44e9-9bd1-220bfd59f759 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:33.320803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc6fa3-e587-44e9-9bd1-220bfd59f759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:33.321661 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc6fa3-e587-44e9-9bd1-220bfd59f759 None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=6bb6d63553794d10a8cba042f8560348) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:33.322003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc6fa3-e587-44e9-9bd1-220bfd59f759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:33.323478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc6fa3-e587-44e9-9bd1-220bfd59f759 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:33.336672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 384/774] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:33 2022] GET /identity/v3/role_assignments?user.id=6bb6d63553794d10a8cba042f8560348&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:33.408806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 391/775] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:33.436356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc6fa3-e587-44e9-9bd1-220bfd59f759 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:33.551366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc6fa3-e587-44e9-9bd1-220bfd59f759 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:33.563000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:33.589685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c5009-80c9-4876-89c6-951a097929c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:33.590524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c5009-80c9-4876-89c6-951a097929c2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:33.590813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c5009-80c9-4876-89c6-951a097929c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:33.591093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c5009-80c9-4876-89c6-951a097929c2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:33.591816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c5009-80c9-4876-89c6-951a097929c2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:33.593258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c5009-80c9-4876-89c6-951a097929c2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-118c5009-80c9-4876-89c6-951a097929c2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:33.605408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:33.606700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:33.607771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 385/776] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:33 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:33.613932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb8b09-e010-4d70-865a-a9f016b489ae None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:33.617259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:33.727484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb8b09-e010-4d70-865a-a9f016b489ae None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:33.762525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0d6a6-9d7d-4ba6-828d-28bff287bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:33.763201 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0d6a6-9d7d-4ba6-828d-28bff287bc51 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:33.763491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0d6a6-9d7d-4ba6-828d-28bff287bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:33.763770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0d6a6-9d7d-4ba6-828d-28bff287bc51 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:33.764485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0d6a6-9d7d-4ba6-828d-28bff287bc51 None admin] RBAC: Adding query filter params (name=network) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:33.764794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0d6a6-9d7d-4ba6-828d-28bff287bc51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:33.766213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0d6a6-9d7d-4ba6-828d-28bff287bc51 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:33.771277 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 392/777] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:02:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:33.776797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c5009-80c9-4876-89c6-951a097929c2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:33.796961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdce2c9-36e9-4a08-9c7c-76222f0eae5b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:33.797273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdce2c9-36e9-4a08-9c7c-76222f0eae5b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:33.797511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdce2c9-36e9-4a08-9c7c-76222f0eae5b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:33.799067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 393/778] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:33.807305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4e10-6cc7-47ab-9935-c4d0f661eae6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:33.807593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4e10-6cc7-47ab-9935-c4d0f661eae6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:33.807878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d4e10-6cc7-47ab-9935-c4d0f661eae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:33.835753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7e6d4e10-6cc7-47ab-9935-c4d0f661eae6 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:33.836965 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7e6d4e10-6cc7-47ab-9935-c4d0f661eae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:33.929342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:33.930882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 394/779] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:33.988699 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d4e10-6cc7-47ab-9935-c4d0f661eae6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:34.056738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c5009-80c9-4876-89c6-951a097929c2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:34.097200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d4e10-6cc7-47ab-9935-c4d0f661eae6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:34.107150 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49665612-63ae-4c90-a0ee-b3cde70b99a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:34.107834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49665612-63ae-4c90-a0ee-b3cde70b99a9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:34.108119 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49665612-63ae-4c90-a0ee-b3cde70b99a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:34.108392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49665612-63ae-4c90-a0ee-b3cde70b99a9 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:34.109118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49665612-63ae-4c90-a0ee-b3cde70b99a9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:34.109424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49665612-63ae-4c90-a0ee-b3cde70b99a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:34.110833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49665612-63ae-4c90-a0ee-b3cde70b99a9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:34.124382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 386/780] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:02:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:34.136268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49665612-63ae-4c90-a0ee-b3cde70b99a9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:34.141208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb1366-fc30-41de-a6f6-d5d392452cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:34.142054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb1366-fc30-41de-a6f6-d5d392452cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:34.142345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb1366-fc30-41de-a6f6-d5d392452cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:34.142623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb1366-fc30-41de-a6f6-d5d392452cc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:34.172123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb1366-fc30-41de-a6f6-d5d392452cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-0ffb1366-fc30-41de-a6f6-d5d392452cc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:34.179894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:34.181294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:34.182212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:34.190052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 395/781] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:34.194590 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:34.196683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb1366-fc30-41de-a6f6-d5d392452cc9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:34.261855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49665612-63ae-4c90-a0ee-b3cde70b99a9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:34.287960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:34.315566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefc904-35fb-41e4-b2fc-654084ae6911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:34.317337 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefc904-35fb-41e4-b2fc-654084ae6911 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:34.317825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefc904-35fb-41e4-b2fc-654084ae6911 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:34.318325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefc904-35fb-41e4-b2fc-654084ae6911 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:34.319357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefc904-35fb-41e4-b2fc-654084ae6911 None admin] RBAC: Adding query filter params (interface=public, service_id=62248bb5690a4a9490225c72b9947db2, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:34.319882 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefc904-35fb-41e4-b2fc-654084ae6911 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:34.321455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefc904-35fb-41e4-b2fc-654084ae6911 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:34.333424 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb1366-fc30-41de-a6f6-d5d392452cc9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:34.339104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 387/782] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 4 09:02:34 2022] GET /identity/v3/endpoints?service_id=62248bb5690a4a9490225c72b9947db2&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:34.345704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefc904-35fb-41e4-b2fc-654084ae6911 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:34.384483 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041b7b0-503c-4bfb-b7ed-8a4f21b738d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:34.385379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041b7b0-503c-4bfb-b7ed-8a4f21b738d6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:34.385495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041b7b0-503c-4bfb-b7ed-8a4f21b738d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:34.385763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041b7b0-503c-4bfb-b7ed-8a4f21b738d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:34.386588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041b7b0-503c-4bfb-b7ed-8a4f21b738d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:34.386932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041b7b0-503c-4bfb-b7ed-8a4f21b738d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:34.388349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041b7b0-503c-4bfb-b7ed-8a4f21b738d6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:34.397655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 396/783] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:34.411527 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041b7b0-503c-4bfb-b7ed-8a4f21b738d6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:34.462364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefc904-35fb-41e4-b2fc-654084ae6911 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:34.466211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:34.512047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6aa19e-87bb-46a4-8ecb-86a8be536ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:34.513407 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aa19e-87bb-46a4-8ecb-86a8be536ebe None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:34.513798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aa19e-87bb-46a4-8ecb-86a8be536ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:34.514210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aa19e-87bb-46a4-8ecb-86a8be536ebe None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:34.515210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6aa19e-87bb-46a4-8ecb-86a8be536ebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:34.516825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6aa19e-87bb-46a4-8ecb-86a8be536ebe None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:34.524683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 388/784] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 4 09:02:34 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:34.623654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041b7b0-503c-4bfb-b7ed-8a4f21b738d6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:34.655051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:34.678720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912d11b-5332-48b0-9cbc-17ea9276b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:34.679559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912d11b-5332-48b0-9cbc-17ea9276b91a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:34.679850 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912d11b-5332-48b0-9cbc-17ea9276b91a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:34.680129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912d11b-5332-48b0-9cbc-17ea9276b91a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:34.693276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912d11b-5332-48b0-9cbc-17ea9276b91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:34.694786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912d11b-5332-48b0-9cbc-17ea9276b91a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f912d11b-5332-48b0-9cbc-17ea9276b91a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:34.717692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:34.718685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:34.719603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:34.719603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:34.719603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:34.719603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:34.719603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 397/785] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:34.726225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6aa19e-87bb-46a4-8ecb-86a8be536ebe None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:34.846716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6aa19e-87bb-46a4-8ecb-86a8be536ebe None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:34.886340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed97a9-8f15-4c1f-a930-fe25e115e4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:34.887014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed97a9-8f15-4c1f-a930-fe25e115e4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:34.887298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed97a9-8f15-4c1f-a930-fe25e115e4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:34.887570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed97a9-8f15-4c1f-a930-fe25e115e4a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:34.888307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed97a9-8f15-4c1f-a930-fe25e115e4a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:34.888653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed97a9-8f15-4c1f-a930-fe25e115e4a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:34.890059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed97a9-8f15-4c1f-a930-fe25e115e4a9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:34.903212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 389/786] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:34.908959 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912d11b-5332-48b0-9cbc-17ea9276b91a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:35.018395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912d11b-5332-48b0-9cbc-17ea9276b91a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:35.025960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534de5d-9837-457b-9d29-d0d2ce95e22a None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.026498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534de5d-9837-457b-9d29-d0d2ce95e22a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.026954 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534de5d-9837-457b-9d29-d0d2ce95e22a None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.028519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 390/787] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:35.033579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424f0d7-7054-48a4-9a10-4defed23a4f5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.033717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424f0d7-7054-48a4-9a10-4defed23a4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.034009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424f0d7-7054-48a4-9a10-4defed23a4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.062533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0966f-3a68-4efe-aa86-e68619f99017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:35.063691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0966f-3a68-4efe-aa86-e68619f99017 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.064221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0966f-3a68-4efe-aa86-e68619f99017 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.064677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0966f-3a68-4efe-aa86-e68619f99017 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.073720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e424f0d7-7054-48a4-9a10-4defed23a4f5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:35.074376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0966f-3a68-4efe-aa86-e68619f99017 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:35.074935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e424f0d7-7054-48a4-9a10-4defed23a4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:35.076249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0966f-3a68-4efe-aa86-e68619f99017 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c1a0966f-3a68-4efe-aa86-e68619f99017 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:35.093667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:35.094709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:35.095662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:35.095662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:35.095662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:35.095662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:35.095662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:35.095662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:35.095662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:35.095662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:35.095662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:35.095662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:35.099349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 398/788] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:35.104680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0966f-3a68-4efe-aa86-e68619f99017 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:35.126241 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:35.188643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:35.242212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 391/789] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:35.291420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424f0d7-7054-48a4-9a10-4defed23a4f5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:35.294880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0966f-3a68-4efe-aa86-e68619f99017 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:35.363720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47c034-5a0d-49ab-bf21-742fe52ca5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:35.364412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47c034-5a0d-49ab-bf21-742fe52ca5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.364696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47c034-5a0d-49ab-bf21-742fe52ca5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.364969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47c034-5a0d-49ab-bf21-742fe52ca5b1 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.365822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47c034-5a0d-49ab-bf21-742fe52ca5b1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:35.366158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47c034-5a0d-49ab-bf21-742fe52ca5b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:35.367623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47c034-5a0d-49ab-bf21-742fe52ca5b1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:35.403074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 399/790] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 4 09:02:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:35.409678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47c034-5a0d-49ab-bf21-742fe52ca5b1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:35.493213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424f0d7-7054-48a4-9a10-4defed23a4f5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:35.560315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a06519-5eb3-47b2-ad54-9a538565132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:35.561576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a06519-5eb3-47b2-ad54-9a538565132d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.562161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a06519-5eb3-47b2-ad54-9a538565132d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.562637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a06519-5eb3-47b2-ad54-9a538565132d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.574634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:35.586360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a06519-5eb3-47b2-ad54-9a538565132d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:35.589717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47c034-5a0d-49ab-bf21-742fe52ca5b1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-92a06519-5eb3-47b2-ad54-9a538565132d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:35.594785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:35.595741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:35.596597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:35.599929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 392/791] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:35.607387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a06519-5eb3-47b2-ad54-9a538565132d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:35.640497 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:35.662583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f444d1-f4ca-4df1-81b3-97a659a53fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:35.663758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f444d1-f4ca-4df1-81b3-97a659a53fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.664250 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f444d1-f4ca-4df1-81b3-97a659a53fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.664758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f444d1-f4ca-4df1-81b3-97a659a53fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.685663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f444d1-f4ca-4df1-81b3-97a659a53fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:35.687179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f444d1-f4ca-4df1-81b3-97a659a53fe5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c4f444d1-f4ca-4df1-81b3-97a659a53fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:35.715748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:35.716726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:35.717639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:35.717639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:35.717639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:35.717639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:35.726051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 400/792] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:35.734096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f444d1-f4ca-4df1-81b3-97a659a53fe5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:35.772870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a06519-5eb3-47b2-ad54-9a538565132d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:35.810524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74000397-847c-4ade-939c-13a6c9c93da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:35.811299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74000397-847c-4ade-939c-13a6c9c93da3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.811685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74000397-847c-4ade-939c-13a6c9c93da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.812061 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74000397-847c-4ade-939c-13a6c9c93da3 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.813028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74000397-847c-4ade-939c-13a6c9c93da3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:35.813545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74000397-847c-4ade-939c-13a6c9c93da3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:35.815042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74000397-847c-4ade-939c-13a6c9c93da3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:35.824901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 393/793] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:35.826406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d602d10-f64f-4a1f-bd56-b8c1d068d353 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.826794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d602d10-f64f-4a1f-bd56-b8c1d068d353 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.827168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d602d10-f64f-4a1f-bd56-b8c1d068d353 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.832634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 394/794] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:35.832634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21294ce8-b98f-4fde-8d0a-a85e46a5df65 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.832634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21294ce8-b98f-4fde-8d0a-a85e46a5df65 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.832634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21294ce8-b98f-4fde-8d0a-a85e46a5df65 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.832935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 395/795] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:35.833643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21294ce8-b98f-4fde-8d0a-a85e46a5df65 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:35.858143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f444d1-f4ca-4df1-81b3-97a659a53fe5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:35.895988 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1039436-6024-41d0-9fd8-29e6420bbc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:35.896779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1039436-6024-41d0-9fd8-29e6420bbc56 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.897098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1039436-6024-41d0-9fd8-29e6420bbc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.897456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1039436-6024-41d0-9fd8-29e6420bbc56 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.898220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1039436-6024-41d0-9fd8-29e6420bbc56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:35.898532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1039436-6024-41d0-9fd8-29e6420bbc56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:35.900025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1039436-6024-41d0-9fd8-29e6420bbc56 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:35.917633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:35.917812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 401/796] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:35.918609 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e994e08-11c0-4695-98b7-03026573a9b5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.918906 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e994e08-11c0-4695-98b7-03026573a9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.919240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e994e08-11c0-4695-98b7-03026573a9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.925669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:35.939416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1e994e08-11c0-4695-98b7-03026573a9b5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:35.940628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1e994e08-11c0-4695-98b7-03026573a9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:35.944526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21294ce8-b98f-4fde-8d0a-a85e46a5df65 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:35.979871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292773d4-0466-409d-8a4b-76647958f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:35.980919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292773d4-0466-409d-8a4b-76647958f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:35.981208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292773d4-0466-409d-8a4b-76647958f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:35.981486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292773d4-0466-409d-8a4b-76647958f9a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:35.993100 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292773d4-0466-409d-8a4b-76647958f9a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:35.994596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292773d4-0466-409d-8a4b-76647958f9a2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-292773d4-0466-409d-8a4b-76647958f9a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:36.017985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:36.018943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:36.019918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:36.019918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:36.019918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:36.019918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:36.020158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 402/797] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:36.020242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 396/798] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:36.020242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292773d4-0466-409d-8a4b-76647958f9a2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:36.020369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd3d8-236b-437d-ab95-8c0196c3276b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:36.020369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd3d8-236b-437d-ab95-8c0196c3276b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:36.020369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd3d8-236b-437d-ab95-8c0196c3276b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:36.040985 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-54abd3d8-236b-437d-ab95-8c0196c3276b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:36.042289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-54abd3d8-236b-437d-ab95-8c0196c3276b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:36.116003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 403/799] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:36.117126 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd3d8-236b-437d-ab95-8c0196c3276b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:36.141403 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292773d4-0466-409d-8a4b-76647958f9a2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:36.174271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:36.181336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376a0ce-70c6-4862-b22b-4dfcb37562b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:36.182305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376a0ce-70c6-4862-b22b-4dfcb37562b5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:36.182591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376a0ce-70c6-4862-b22b-4dfcb37562b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:36.182867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376a0ce-70c6-4862-b22b-4dfcb37562b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:36.190266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376a0ce-70c6-4862-b22b-4dfcb37562b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:36.192031 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376a0ce-70c6-4862-b22b-4dfcb37562b5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:36.193116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-e376a0ce-70c6-4862-b22b-4dfcb37562b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:36.209051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:36.210180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:36.211117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:36.211117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:36.211117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:36.211117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:36.211117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:36.211117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:36.211117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:36.211117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:36.211117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 397/800] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:36.212124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376a0ce-70c6-4862-b22b-4dfcb37562b5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:36.241362 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd3d8-236b-437d-ab95-8c0196c3276b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:36.278043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6d156-481b-4ecd-807a-2b83cc94bbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:36.278701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d156-481b-4ecd-807a-2b83cc94bbad None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:36.279037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d156-481b-4ecd-807a-2b83cc94bbad None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:36.279390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d156-481b-4ecd-807a-2b83cc94bbad None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:36.280118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6d156-481b-4ecd-807a-2b83cc94bbad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:36.280440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6d156-481b-4ecd-807a-2b83cc94bbad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:36.281821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6d156-481b-4ecd-807a-2b83cc94bbad None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:36.297478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 404/801] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:36.297478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6d156-481b-4ecd-807a-2b83cc94bbad None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:36.320783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376a0ce-70c6-4862-b22b-4dfcb37562b5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:36.355000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c080e51-6a60-4771-b2d3-9c430fe22925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:36.355808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c080e51-6a60-4771-b2d3-9c430fe22925 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:36.356093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c080e51-6a60-4771-b2d3-9c430fe22925 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:36.356376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c080e51-6a60-4771-b2d3-9c430fe22925 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:36.357037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c080e51-6a60-4771-b2d3-9c430fe22925 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:36.358444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c080e51-6a60-4771-b2d3-9c430fe22925 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5c080e51-6a60-4771-b2d3-9c430fe22925 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:36.371014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:36.371940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:36.372798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 398/802] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jul 4 09:02:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:36.374130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c080e51-6a60-4771-b2d3-9c430fe22925 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:36.411593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6d156-481b-4ecd-807a-2b83cc94bbad None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:36.430977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:36.435142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:36.447667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7db86d-2584-48b6-ba95-9ef7394f1a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:36.448469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db86d-2584-48b6-ba95-9ef7394f1a24 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:36.448747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db86d-2584-48b6-ba95-9ef7394f1a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:36.449020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db86d-2584-48b6-ba95-9ef7394f1a24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:36.459050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7db86d-2584-48b6-ba95-9ef7394f1a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2d7db86d-2584-48b6-ba95-9ef7394f1a24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:36.466704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:36.467691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:36.468551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 405/803] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:36.469257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7db86d-2584-48b6-ba95-9ef7394f1a24 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:36.485224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c080e51-6a60-4771-b2d3-9c430fe22925 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:36.521289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f28d9-4c9f-46f3-b251-322cfb7595cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:36.522076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f28d9-4c9f-46f3-b251-322cfb7595cc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:36.522264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f28d9-4c9f-46f3-b251-322cfb7595cc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:36.522539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f28d9-4c9f-46f3-b251-322cfb7595cc None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:36.523339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f28d9-4c9f-46f3-b251-322cfb7595cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:36.523646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f28d9-4c9f-46f3-b251-322cfb7595cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:36.525094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f28d9-4c9f-46f3-b251-322cfb7595cc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:36.541921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 399/804] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:36.541921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f28d9-4c9f-46f3-b251-322cfb7595cc None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:36.583188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7db86d-2584-48b6-ba95-9ef7394f1a24 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:36.618733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b06bb6-1430-475f-a5e5-fe5d7d8fa72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:36.620075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b06bb6-1430-475f-a5e5-fe5d7d8fa72b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:36.620561 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b06bb6-1430-475f-a5e5-fe5d7d8fa72b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:36.621091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b06bb6-1430-475f-a5e5-fe5d7d8fa72b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:36.629979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b06bb6-1430-475f-a5e5-fe5d7d8fa72b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:36.631388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b06bb6-1430-475f-a5e5-fe5d7d8fa72b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:36.644624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f28d9-4c9f-46f3-b251-322cfb7595cc None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-72b06bb6-1430-475f-a5e5-fe5d7d8fa72b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:36.646672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:36.647648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:36.648603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:36.648603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:36.648603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:36.648603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:36.648603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:36.648603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:36.648603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:36.648603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:36.648603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:02:36.648603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:36.652507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 406/805] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 4 09:02:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:36.653168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b06bb6-1430-475f-a5e5-fe5d7d8fa72b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:36.664285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:36.685005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20616a8a-9e5d-43ae-9bde-eb7965159348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:36.685680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20616a8a-9e5d-43ae-9bde-eb7965159348 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:36.685987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20616a8a-9e5d-43ae-9bde-eb7965159348 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:36.686266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20616a8a-9e5d-43ae-9bde-eb7965159348 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:36.686980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20616a8a-9e5d-43ae-9bde-eb7965159348 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:36.687288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20616a8a-9e5d-43ae-9bde-eb7965159348 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:36.688857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20616a8a-9e5d-43ae-9bde-eb7965159348 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:36.693636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 400/806] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 4 09:02:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:36.694352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20616a8a-9e5d-43ae-9bde-eb7965159348 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:36.707797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:36.768154 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b06bb6-1430-475f-a5e5-fe5d7d8fa72b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:36.797397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20616a8a-9e5d-43ae-9bde-eb7965159348 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:36.803728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630f0f0-ece8-4695-a8d8-13468eaf3565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:36.804416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630f0f0-ece8-4695-a8d8-13468eaf3565 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:36.804703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630f0f0-ece8-4695-a8d8-13468eaf3565 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:36.804979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630f0f0-ece8-4695-a8d8-13468eaf3565 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:36.805740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630f0f0-ece8-4695-a8d8-13468eaf3565 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:36.806077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630f0f0-ece8-4695-a8d8-13468eaf3565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:36.807459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630f0f0-ece8-4695-a8d8-13468eaf3565 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:36.815454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 407/807] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:36.816448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630f0f0-ece8-4695-a8d8-13468eaf3565 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:36.839413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7b415-8bf2-432d-a110-b1dc522d8b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:36.840621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7b415-8bf2-432d-a110-b1dc522d8b4d None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:36.841101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7b415-8bf2-432d-a110-b1dc522d8b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:36.841638 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7b415-8bf2-432d-a110-b1dc522d8b4d None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/8cbb65dbfadf4c63a272f5318efa875a/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:36.843934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7b415-8bf2-432d-a110-b1dc522d8b4d None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=8cbb65dbfadf4c63a272f5318efa875a, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:36.845741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7b415-8bf2-432d-a110-b1dc522d8b4d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:36.870036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 401/808] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:02:36 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/8cbb65dbfadf4c63a272f5318efa875a/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:02:36.871217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7b415-8bf2-432d-a110-b1dc522d8b4d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:36.922731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:36.936619 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630f0f0-ece8-4695-a8d8-13468eaf3565 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:36.964522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:36.986033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b716676-6aa2-4182-af13-06eb19a04645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:36.986573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b716676-6aa2-4182-af13-06eb19a04645 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:36.986861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b716676-6aa2-4182-af13-06eb19a04645 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:36.987136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b716676-6aa2-4182-af13-06eb19a04645 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:36.988084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b716676-6aa2-4182-af13-06eb19a04645 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:36.988543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b716676-6aa2-4182-af13-06eb19a04645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:36.990080 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b716676-6aa2-4182-af13-06eb19a04645 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:37.009037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7b415-8bf2-432d-a110-b1dc522d8b4d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:37.014852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 408/809] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 4 09:02:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:37.014852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b716676-6aa2-4182-af13-06eb19a04645 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:37.055927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb599-f472-4667-be87-175f6f51aee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:37.056604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb599-f472-4667-be87-175f6f51aee3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:37.056894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb599-f472-4667-be87-175f6f51aee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:37.057170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb599-f472-4667-be87-175f6f51aee3 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:37.057942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ceb599-f472-4667-be87-175f6f51aee3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:37.058306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ceb599-f472-4667-be87-175f6f51aee3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:37.059683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ceb599-f472-4667-be87-175f6f51aee3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:37.065171 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 402/810] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 4 09:02:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:37.065831 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb599-f472-4667-be87-175f6f51aee3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:37.133837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b716676-6aa2-4182-af13-06eb19a04645 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:37.173785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69965102-f775-4751-90f1-068811d058fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:37.174633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965102-f775-4751-90f1-068811d058fe None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:37.174920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965102-f775-4751-90f1-068811d058fe None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:37.175196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965102-f775-4751-90f1-068811d058fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:37.176369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb599-f472-4667-be87-175f6f51aee3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:37.178243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:37.187751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69965102-f775-4751-90f1-068811d058fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:37.189235 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69965102-f775-4751-90f1-068811d058fe None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-69965102-f775-4751-90f1-068811d058fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:37.209597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:37.216437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:37.217349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:37.217349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:37.217349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:37.217349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:37.217349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 409/811] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:37.217349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44f2e-7477-4869-916a-4aa72cb88595 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:37.217349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44f2e-7477-4869-916a-4aa72cb88595 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:37.217349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44f2e-7477-4869-916a-4aa72cb88595 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:37.217349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 410/812] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:37.218399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44f2e-7477-4869-916a-4aa72cb88595 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:37.228857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160f288-4203-4b1b-ba51-f704e4c731d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:37.229616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160f288-4203-4b1b-ba51-f704e4c731d0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:37.229995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160f288-4203-4b1b-ba51-f704e4c731d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:37.230352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160f288-4203-4b1b-ba51-f704e4c731d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:37.231284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:37.305971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160f288-4203-4b1b-ba51-f704e4c731d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:37.307731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160f288-4203-4b1b-ba51-f704e4c731d0 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:37.324985 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44f2e-7477-4869-916a-4aa72cb88595 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:37.364520 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae398e9d-7705-4199-abec-5cd2cc7b39e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:37.365182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae398e9d-7705-4199-abec-5cd2cc7b39e0 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:37.365463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae398e9d-7705-4199-abec-5cd2cc7b39e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:37.365738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae398e9d-7705-4199-abec-5cd2cc7b39e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:37.366607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae398e9d-7705-4199-abec-5cd2cc7b39e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:37.368157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae398e9d-7705-4199-abec-5cd2cc7b39e0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:37.401420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 411/813] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 4 09:02:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:37.401420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae398e9d-7705-4199-abec-5cd2cc7b39e0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:37.423083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 403/814] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:02:37.429583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccee67-e679-46c7-a477-69f8a93a194f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:37.429971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccee67-e679-46c7-a477-69f8a93a194f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:37.430333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccee67-e679-46c7-a477-69f8a93a194f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:37.435773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:37.449934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9fccee67-e679-46c7-a477-69f8a93a194f None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:37.451257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9fccee67-e679-46c7-a477-69f8a93a194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:37.500532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:37.540642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 404/815] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:37.541915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fccee67-e679-46c7-a477-69f8a93a194f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:37.549537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae398e9d-7705-4199-abec-5cd2cc7b39e0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:37.588560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe71024-50df-402b-b0f6-0e6cbadcb860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:37.589554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe71024-50df-402b-b0f6-0e6cbadcb860 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:37.590078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe71024-50df-402b-b0f6-0e6cbadcb860 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:37.590546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe71024-50df-402b-b0f6-0e6cbadcb860 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:37.591448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe71024-50df-402b-b0f6-0e6cbadcb860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:37.591927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe71024-50df-402b-b0f6-0e6cbadcb860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:37.594223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe71024-50df-402b-b0f6-0e6cbadcb860 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:37.611838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 412/816] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:37.613169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe71024-50df-402b-b0f6-0e6cbadcb860 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:37.651561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fccee67-e679-46c7-a477-69f8a93a194f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:37.692363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:37.696038 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1349db-760c-4ed2-8342-23e943107cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:37.697109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1349db-760c-4ed2-8342-23e943107cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:37.697462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1349db-760c-4ed2-8342-23e943107cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:37.698249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1349db-760c-4ed2-8342-23e943107cf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:37.709272 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1349db-760c-4ed2-8342-23e943107cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:37.710966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1349db-760c-4ed2-8342-23e943107cf5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:37.730403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe71024-50df-402b-b0f6-0e6cbadcb860 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-eb1349db-760c-4ed2-8342-23e943107cf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:37.733344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:37.734242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:37.735157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:37.735157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:37.735157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:37.735157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:37.736998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 405/817] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:37.738176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1349db-760c-4ed2-8342-23e943107cf5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:37.772831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e417e36-859a-45b7-b2e0-c6f4cdd419e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:37.773636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e417e36-859a-45b7-b2e0-c6f4cdd419e8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:37.773976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e417e36-859a-45b7-b2e0-c6f4cdd419e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:37.774261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e417e36-859a-45b7-b2e0-c6f4cdd419e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:37.775072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e417e36-859a-45b7-b2e0-c6f4cdd419e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:37.777018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e417e36-859a-45b7-b2e0-c6f4cdd419e8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-3e417e36-859a-45b7-b2e0-c6f4cdd419e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:37.790957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:37.791990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:37.793229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 413/818] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:37 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:37.794327 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e417e36-859a-45b7-b2e0-c6f4cdd419e8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:37.794327 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:37.871005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1349db-760c-4ed2-8342-23e943107cf5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:37.914418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cff559-d2ce-44dd-8975-96f989e36aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:37.915133 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cff559-d2ce-44dd-8975-96f989e36aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:37.915414 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cff559-d2ce-44dd-8975-96f989e36aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:37.915686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cff559-d2ce-44dd-8975-96f989e36aa7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:37.923395 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cff559-d2ce-44dd-8975-96f989e36aa7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:37.924640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e417e36-859a-45b7-b2e0-c6f4cdd419e8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:37.924857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cff559-d2ce-44dd-8975-96f989e36aa7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-b0cff559-d2ce-44dd-8975-96f989e36aa7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:37.940376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:37.941515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:37.942527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:37.942527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:37.942527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:37.942527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:37.942527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:37.942527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:37.942527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:37.942527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:37.942527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:37.942527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:37.946427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 406/819] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 4 09:02:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:37.947315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cff559-d2ce-44dd-8975-96f989e36aa7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:37.971545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9d87e-37de-481c-9390-bf899025cf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:37.973029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9d87e-37de-481c-9390-bf899025cf69 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:37.973551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9d87e-37de-481c-9390-bf899025cf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:37.974048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9d87e-37de-481c-9390-bf899025cf69 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:37.975821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9d87e-37de-481c-9390-bf899025cf69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:37.976372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9d87e-37de-481c-9390-bf899025cf69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:37.978514 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9d87e-37de-481c-9390-bf899025cf69 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:37.995629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:37.999566 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 414/820] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:38.001559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9d87e-37de-481c-9390-bf899025cf69 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:38.065536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cff559-d2ce-44dd-8975-96f989e36aa7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:38.083377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:38.105345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf1c6b-e45a-453c-81b2-71db45d0b231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:38.106191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf1c6b-e45a-453c-81b2-71db45d0b231 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:38.106583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf1c6b-e45a-453c-81b2-71db45d0b231 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:38.106957 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf1c6b-e45a-453c-81b2-71db45d0b231 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:38.107931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf1c6b-e45a-453c-81b2-71db45d0b231 None admin] RBAC: Adding query filter params (name=network) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:38.108355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf1c6b-e45a-453c-81b2-71db45d0b231 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:38.109990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf1c6b-e45a-453c-81b2-71db45d0b231 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:38.115440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 407/821] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:02:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:38.116491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf1c6b-e45a-453c-81b2-71db45d0b231 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:38.134740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9d87e-37de-481c-9390-bf899025cf69 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:38.172531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2b995-b99b-4285-8ad9-db42b44263fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:38.173517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2b995-b99b-4285-8ad9-db42b44263fc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:38.174029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2b995-b99b-4285-8ad9-db42b44263fc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:38.174536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2b995-b99b-4285-8ad9-db42b44263fc None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:38.175593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2b995-b99b-4285-8ad9-db42b44263fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:38.176184 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2b995-b99b-4285-8ad9-db42b44263fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:38.177818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2b995-b99b-4285-8ad9-db42b44263fc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:38.200148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 415/822] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 4 09:02:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:38.201352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2b995-b99b-4285-8ad9-db42b44263fc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:38.227996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf1c6b-e45a-453c-81b2-71db45d0b231 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:38.230656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:38.269673 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538dfa0-74f9-49c3-a57d-44547b31192a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:38.270587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538dfa0-74f9-49c3-a57d-44547b31192a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:38.270922 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538dfa0-74f9-49c3-a57d-44547b31192a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:38.271257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538dfa0-74f9-49c3-a57d-44547b31192a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:38.278919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538dfa0-74f9-49c3-a57d-44547b31192a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:38.280469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538dfa0-74f9-49c3-a57d-44547b31192a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5538dfa0-74f9-49c3-a57d-44547b31192a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:38.296603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:38.297543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:38.298528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:38.298528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:38.298528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:38.298528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:38.298528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:38.298528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:38.298528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:38.298528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:38.299195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 408/823] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:38.300172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538dfa0-74f9-49c3-a57d-44547b31192a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:38.305987 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2b995-b99b-4285-8ad9-db42b44263fc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:38.320025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:38.344073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812ed34-3a61-4326-b65d-2cd3758e0a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:38.344989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812ed34-3a61-4326-b65d-2cd3758e0a8d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:38.345413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812ed34-3a61-4326-b65d-2cd3758e0a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:38.345859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812ed34-3a61-4326-b65d-2cd3758e0a8d None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:38.346734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812ed34-3a61-4326-b65d-2cd3758e0a8d None admin] RBAC: Adding query filter params (type=network) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:38.347196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812ed34-3a61-4326-b65d-2cd3758e0a8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:38.348818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812ed34-3a61-4326-b65d-2cd3758e0a8d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:38.354598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 416/824] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:02:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:38.355660 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812ed34-3a61-4326-b65d-2cd3758e0a8d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:38.415109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538dfa0-74f9-49c3-a57d-44547b31192a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:38.453761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebadfb-2e58-4499-93d0-79548437d4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:38.454693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebadfb-2e58-4499-93d0-79548437d4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:38.455078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebadfb-2e58-4499-93d0-79548437d4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:38.455457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebadfb-2e58-4499-93d0-79548437d4d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:38.456312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812ed34-3a61-4326-b65d-2cd3758e0a8d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:38.467581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebadfb-2e58-4499-93d0-79548437d4d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:38.469140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebadfb-2e58-4499-93d0-79548437d4d8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-e5ebadfb-2e58-4499-93d0-79548437d4d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:38.488965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:38.490044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:38.490974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:38.490974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:38.490974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:38.490974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:38.493878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e53b6b-62f3-4766-a03c-a61156bdf5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:38.494151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 409/825] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:38.495015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebadfb-2e58-4499-93d0-79548437d4d8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:38.495133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53b6b-62f3-4766-a03c-a61156bdf5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:38.495676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53b6b-62f3-4766-a03c-a61156bdf5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:38.496154 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53b6b-62f3-4766-a03c-a61156bdf5d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:38.497491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e53b6b-62f3-4766-a03c-a61156bdf5d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:38.498031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e53b6b-62f3-4766-a03c-a61156bdf5d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:38.499677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e53b6b-62f3-4766-a03c-a61156bdf5d3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:38.512920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 417/826] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:38.514138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e53b6b-62f3-4766-a03c-a61156bdf5d3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:38.516250 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:38.550010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:38.612149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebadfb-2e58-4499-93d0-79548437d4d8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:38.616115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e53b6b-62f3-4766-a03c-a61156bdf5d3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:38.651487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0d39f-8dba-49ad-8713-45fd4537f4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:38.652429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a5da-5f1a-4182-b8db-d235a76c53e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:38.652583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0d39f-8dba-49ad-8713-45fd4537f4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:38.653015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0d39f-8dba-49ad-8713-45fd4537f4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:38.653366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a5da-5f1a-4182-b8db-d235a76c53e9 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:38.653593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0d39f-8dba-49ad-8713-45fd4537f4e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:38.653830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a5da-5f1a-4182-b8db-d235a76c53e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:38.654261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a5da-5f1a-4182-b8db-d235a76c53e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:38.654823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0d39f-8dba-49ad-8713-45fd4537f4e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:38.655214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1a5da-5f1a-4182-b8db-d235a76c53e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:38.655412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0d39f-8dba-49ad-8713-45fd4537f4e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:38.656791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1a5da-5f1a-4182-b8db-d235a76c53e9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:38.657145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0d39f-8dba-49ad-8713-45fd4537f4e2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:38.670685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 418/827] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:38.672095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0d39f-8dba-49ad-8713-45fd4537f4e2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:38.685942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 410/828] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 4 09:02:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:38.687009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a5da-5f1a-4182-b8db-d235a76c53e9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:38.770911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:38.773963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:38.780355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0d39f-8dba-49ad-8713-45fd4537f4e2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:38.824550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a5da-5f1a-4182-b8db-d235a76c53e9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:38.824755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dadacd-d310-4ed7-b324-bf76e53ceb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:38.825321 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dadacd-d310-4ed7-b324-bf76e53ceb2b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:38.825656 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dadacd-d310-4ed7-b324-bf76e53ceb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:38.825967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dadacd-d310-4ed7-b324-bf76e53ceb2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:38.826871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dadacd-d310-4ed7-b324-bf76e53ceb2b None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=739f3b5eaef542c29a0cf0b475c0d965) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:38.827194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dadacd-d310-4ed7-b324-bf76e53ceb2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:38.830037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dadacd-d310-4ed7-b324-bf76e53ceb2b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:38.839137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 419/829] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:38 2022] GET /identity/v3/role_assignments?user.id=739f3b5eaef542c29a0cf0b475c0d965&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=2302517080ac42578aa81c606dd0ff2f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:38.890506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d23b9-332c-41b3-9d05-c374a25062fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:38.891792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d23b9-332c-41b3-9d05-c374a25062fb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:38.892293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d23b9-332c-41b3-9d05-c374a25062fb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:38.892766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d23b9-332c-41b3-9d05-c374a25062fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:38.907267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d23b9-332c-41b3-9d05-c374a25062fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:38.909179 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d23b9-332c-41b3-9d05-c374a25062fb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-013d23b9-332c-41b3-9d05-c374a25062fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:38.927904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:38.928876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:38.929870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:38.929870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:38.929870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:38.929870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:38.929870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:38.929870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:38.929870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:38.929870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:38.933191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 411/830] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:38.938958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dadacd-d310-4ed7-b324-bf76e53ceb2b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:39.029670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e0ebe-5aef-4f93-988d-64b2f51ee008 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:39.029987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e0ebe-5aef-4f93-988d-64b2f51ee008 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:39.030174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e0ebe-5aef-4f93-988d-64b2f51ee008 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:39.031683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 412/831] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:39.037794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181301f-1e59-4792-8611-04202b11e37b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:39.038412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181301f-1e59-4792-8611-04202b11e37b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:39.038412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181301f-1e59-4792-8611-04202b11e37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:39.059522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4181301f-1e59-4792-8611-04202b11e37b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:39.060703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4181301f-1e59-4792-8611-04202b11e37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:39.063059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dadacd-d310-4ed7-b324-bf76e53ceb2b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:39.106478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3d675-d1bd-4f49-abe4-1482ff67f959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:39.107157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3d675-d1bd-4f49-abe4-1482ff67f959 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:39.107440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3d675-d1bd-4f49-abe4-1482ff67f959 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:39.107716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3d675-d1bd-4f49-abe4-1482ff67f959 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:39.108519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3d675-d1bd-4f49-abe4-1482ff67f959 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:39.108830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3d675-d1bd-4f49-abe4-1482ff67f959 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:39.110343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3d675-d1bd-4f49-abe4-1482ff67f959 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:39.123976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 420/832] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:39.130418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3d675-d1bd-4f49-abe4-1482ff67f959 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:39.141104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 413/833] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:02:39.158633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:39.164928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181301f-1e59-4792-8611-04202b11e37b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:39.237381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:39.265797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3d675-d1bd-4f49-abe4-1482ff67f959 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:39.287315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181301f-1e59-4792-8611-04202b11e37b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:39.308421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fb452-a7cf-460f-a837-da22f35b49be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:39.309321 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fb452-a7cf-460f-a837-da22f35b49be None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:39.309604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fb452-a7cf-460f-a837-da22f35b49be None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:39.309876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fb452-a7cf-460f-a837-da22f35b49be None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/6bb6d63553794d10a8cba042f8560348/roles/f0e6d5b558954227a2e1423c3d1e5d18` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:39.330619 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1fb452-a7cf-460f-a837-da22f35b49be None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=6bb6d63553794d10a8cba042f8560348, role_id=f0e6d5b558954227a2e1423c3d1e5d18)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:39.333043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25e838-d5a3-420b-ae05-293292f26af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:39.333850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25e838-d5a3-420b-ae05-293292f26af7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:39.334149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25e838-d5a3-420b-ae05-293292f26af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:39.334416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25e838-d5a3-420b-ae05-293292f26af7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:39.335236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1fb452-a7cf-460f-a837-da22f35b49be None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:39.349792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25e838-d5a3-420b-ae05-293292f26af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:39.352762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 421/834] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:02:39 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/6bb6d63553794d10a8cba042f8560348/roles/f0e6d5b558954227a2e1423c3d1e5d18 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5b25e838-d5a3-420b-ae05-293292f26af7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:39.362026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:39.363170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:02:39.364057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:39.365280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 414/835] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:39.366446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fb452-a7cf-460f-a837-da22f35b49be None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:39.448328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:39.483818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fb452-a7cf-460f-a837-da22f35b49be None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:39.523573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e893be7-9a22-437b-b99e-ed49e7f91426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:39.524319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e893be7-9a22-437b-b99e-ed49e7f91426 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:39.524609 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e893be7-9a22-437b-b99e-ed49e7f91426 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:39.524894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e893be7-9a22-437b-b99e-ed49e7f91426 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:39.525661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e893be7-9a22-437b-b99e-ed49e7f91426 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:39.525996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e893be7-9a22-437b-b99e-ed49e7f91426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:39.527474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e893be7-9a22-437b-b99e-ed49e7f91426 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:39.537142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 422/836] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:39.543042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25e838-d5a3-420b-ae05-293292f26af7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:39.644390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc38773-d53e-46a3-9dfc-12e1312f84d2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:39.644656 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc38773-d53e-46a3-9dfc-12e1312f84d2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:39.645635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc38773-d53e-46a3-9dfc-12e1312f84d2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:39.646749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 423/837] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:39.651518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbc8d1-db09-4d72-ae7b-5279b98442f7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:39.651807 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbc8d1-db09-4d72-ae7b-5279b98442f7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:39.652093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbc8d1-db09-4d72-ae7b-5279b98442f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:39.657158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25e838-d5a3-420b-ae05-293292f26af7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:39.669728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bccbc8d1-db09-4d72-ae7b-5279b98442f7 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:39.671075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bccbc8d1-db09-4d72-ae7b-5279b98442f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:39.678069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:39.696343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc7f3a-c370-4d47-ab55-25d679db968d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:39.697150 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc7f3a-c370-4d47-ab55-25d679db968d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:39.697435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc7f3a-c370-4d47-ab55-25d679db968d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:39.697712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc7f3a-c370-4d47-ab55-25d679db968d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:39.711690 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc7f3a-c370-4d47-ab55-25d679db968d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:39.713155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc7f3a-c370-4d47-ab55-25d679db968d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ffcc7f3a-c370-4d47-ab55-25d679db968d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:39.731774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:39.732728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:39.733628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:39.733628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:39.733628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:39.733628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:39.733628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 415/838] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:39.738858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc7f3a-c370-4d47-ab55-25d679db968d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:39.747303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 424/839] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:02:39.773448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbc8d1-db09-4d72-ae7b-5279b98442f7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:39.846098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc7f3a-c370-4d47-ab55-25d679db968d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:39.847258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:39.888125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8575e6-48e0-4ae6-9531-1c95461d8a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:39.889078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8575e6-48e0-4ae6-9531-1c95461d8a67 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:39.889506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8575e6-48e0-4ae6-9531-1c95461d8a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:39.889947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8575e6-48e0-4ae6-9531-1c95461d8a67 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:39.890809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8575e6-48e0-4ae6-9531-1c95461d8a67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:39.891256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8575e6-48e0-4ae6-9531-1c95461d8a67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:39.892782 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8575e6-48e0-4ae6-9531-1c95461d8a67 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:39.901649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbc8d1-db09-4d72-ae7b-5279b98442f7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:39.905456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 416/840] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:39.911077 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8575e6-48e0-4ae6-9531-1c95461d8a67 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:39.941800 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:39.948006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fe047-b1f7-4940-9c1c-a1fe3ec2345a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:39.949279 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe047-b1f7-4940-9c1c-a1fe3ec2345a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:39.949747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe047-b1f7-4940-9c1c-a1fe3ec2345a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:39.950216 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe047-b1f7-4940-9c1c-a1fe3ec2345a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:39.951045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fe047-b1f7-4940-9c1c-a1fe3ec2345a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:39.953681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fe047-b1f7-4940-9c1c-a1fe3ec2345a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e88fe047-b1f7-4940-9c1c-a1fe3ec2345a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:39.971382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:39.972445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:39.973278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 425/841] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:02:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:39.982914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fe047-b1f7-4940-9c1c-a1fe3ec2345a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:40.016442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8575e6-48e0-4ae6-9531-1c95461d8a67 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:40.055059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d473ba-75b4-4ce9-9803-f680122b34ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:40.056218 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d473ba-75b4-4ce9-9803-f680122b34ca None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:40.056839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d473ba-75b4-4ce9-9803-f680122b34ca None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:40.057301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d473ba-75b4-4ce9-9803-f680122b34ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:40.065331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d473ba-75b4-4ce9-9803-f680122b34ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:40.067529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d473ba-75b4-4ce9-9803-f680122b34ca None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-42d473ba-75b4-4ce9-9803-f680122b34ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:40.083556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:40.084579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:40.085553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:40.085553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:40.085553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:40.085553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:40.085553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:40.085553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:40.085553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:40.085553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:40.085553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 04 09:02:40.085553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:40.088490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 417/842] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:02:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:40.099180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d473ba-75b4-4ce9-9803-f680122b34ca None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:40.208361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:40.221250 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d473ba-75b4-4ce9-9803-f680122b34ca None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:40.235497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fe047-b1f7-4940-9c1c-a1fe3ec2345a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:40.258960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e57cd-7b09-4aa0-8754-4e02685ebcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:40.259641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e57cd-7b09-4aa0-8754-4e02685ebcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:40.259975 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e57cd-7b09-4aa0-8754-4e02685ebcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:40.260462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e57cd-7b09-4aa0-8754-4e02685ebcc6 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:40.261100 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e57cd-7b09-4aa0-8754-4e02685ebcc6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:40.261410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e57cd-7b09-4aa0-8754-4e02685ebcc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:40.262896 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e57cd-7b09-4aa0-8754-4e02685ebcc6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:40.275411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:40.287050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 418/843] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 4 09:02:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:40.288105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e57cd-7b09-4aa0-8754-4e02685ebcc6 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:40.291713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbb3f2-805f-4b0f-9bdd-76572f8730db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:40.292392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbb3f2-805f-4b0f-9bdd-76572f8730db None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:40.292675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbb3f2-805f-4b0f-9bdd-76572f8730db None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:40.292952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbb3f2-805f-4b0f-9bdd-76572f8730db None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:40.294218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbb3f2-805f-4b0f-9bdd-76572f8730db None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:40.294218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbb3f2-805f-4b0f-9bdd-76572f8730db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:40.295347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbb3f2-805f-4b0f-9bdd-76572f8730db None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:40.304910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 426/844] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:02:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:40.307352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbb3f2-805f-4b0f-9bdd-76572f8730db None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:40.402194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e57cd-7b09-4aa0-8754-4e02685ebcc6 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:40.422493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbb3f2-805f-4b0f-9bdd-76572f8730db None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:40.438395 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53429746-4eaa-4865-bba3-7a018e34b70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:40.439185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429746-4eaa-4865-bba3-7a018e34b70c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:40.439587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429746-4eaa-4865-bba3-7a018e34b70c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:40.439962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429746-4eaa-4865-bba3-7a018e34b70c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:40.459433 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:40.460542 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0dd43-1731-4359-acb3-a1683978543d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:40.461510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0dd43-1731-4359-acb3-a1683978543d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:40.462149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0dd43-1731-4359-acb3-a1683978543d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:40.462468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0dd43-1731-4359-acb3-a1683978543d None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:40.463354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0dd43-1731-4359-acb3-a1683978543d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:40.463837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0dd43-1731-4359-acb3-a1683978543d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:40.466382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0dd43-1731-4359-acb3-a1683978543d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:40.473896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 427/845] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:02:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:40.518450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53429746-4eaa-4865-bba3-7a018e34b70c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:40.520053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53429746-4eaa-4865-bba3-7a018e34b70c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:40.632243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 419/846] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:40 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 09:02:40.637945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0dd43-1731-4359-acb3-a1683978543d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:40.696435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:40.742194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0dd43-1731-4359-acb3-a1683978543d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:40.776518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400ea09-f297-4eea-b30e-161f14bbf71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:40.777315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400ea09-f297-4eea-b30e-161f14bbf71c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:40.777592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400ea09-f297-4eea-b30e-161f14bbf71c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:40.777859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400ea09-f297-4eea-b30e-161f14bbf71c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:40.788830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400ea09-f297-4eea-b30e-161f14bbf71c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:40.790304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400ea09-f297-4eea-b30e-161f14bbf71c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-0400ea09-f297-4eea-b30e-161f14bbf71c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:40.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:40.808318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:40.809231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:40.809231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:40.809231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:40.809231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:40.809231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 428/847] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:40.815657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53429746-4eaa-4865-bba3-7a018e34b70c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:40.867217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:40.923032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53429746-4eaa-4865-bba3-7a018e34b70c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:40.960912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db575a-356a-416d-9bc2-3a430ef816dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:40.961496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db575a-356a-416d-9bc2-3a430ef816dd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:40.961777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db575a-356a-416d-9bc2-3a430ef816dd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:40.962084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db575a-356a-416d-9bc2-3a430ef816dd None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:40.962810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db575a-356a-416d-9bc2-3a430ef816dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:40.963113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db575a-356a-416d-9bc2-3a430ef816dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:40.964475 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db575a-356a-416d-9bc2-3a430ef816dd None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:40.976894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 420/848] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:40.982080 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400ea09-f297-4eea-b30e-161f14bbf71c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:41.084685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400ea09-f297-4eea-b30e-161f14bbf71c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:41.092216 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:41.119047 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994c4df-9e95-430f-abf7-29893d4e86d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:41.119851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994c4df-9e95-430f-abf7-29893d4e86d4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:41.120131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994c4df-9e95-430f-abf7-29893d4e86d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:41.120400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994c4df-9e95-430f-abf7-29893d4e86d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:41.128517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994c4df-9e95-430f-abf7-29893d4e86d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:41.130647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994c4df-9e95-430f-abf7-29893d4e86d4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9994c4df-9e95-430f-abf7-29893d4e86d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:41.143800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:41.144801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:41.145725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:41.145725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:41.145725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:41.145725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:41.145725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:41.145725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:41.145725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:41.145725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:41.145725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 429/849] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:41.157621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db575a-356a-416d-9bc2-3a430ef816dd None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:41.170618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd92e-8bb5-4495-8fc9-a6f5479522c3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:41.170896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd92e-8bb5-4495-8fc9-a6f5479522c3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:41.171169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd92e-8bb5-4495-8fc9-a6f5479522c3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:41.172862 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 430/850] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:41.177814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501edab4-f85e-421a-8fd5-7fa3832f3272 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:41.178153 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501edab4-f85e-421a-8fd5-7fa3832f3272 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:41.178423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501edab4-f85e-421a-8fd5-7fa3832f3272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:41.196519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-501edab4-f85e-421a-8fd5-7fa3832f3272 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:41.197703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-501edab4-f85e-421a-8fd5-7fa3832f3272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:41.267927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db575a-356a-416d-9bc2-3a430ef816dd None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:41.274147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 431/851] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:41.277553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:41.293133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501edab4-f85e-421a-8fd5-7fa3832f3272 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:41.304048 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dc54c-240a-4075-8d55-2c47a9a9ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:41.304778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dc54c-240a-4075-8d55-2c47a9a9ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:41.305089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dc54c-240a-4075-8d55-2c47a9a9ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:41.305431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dc54c-240a-4075-8d55-2c47a9a9ecb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:41.306307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dc54c-240a-4075-8d55-2c47a9a9ecb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:41.306716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dc54c-240a-4075-8d55-2c47a9a9ecb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:41.308260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dc54c-240a-4075-8d55-2c47a9a9ecb4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:41.322356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 421/852] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:41.328007 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dc54c-240a-4075-8d55-2c47a9a9ecb4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:41.400058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:41.408819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501edab4-f85e-421a-8fd5-7fa3832f3272 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:41.452537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dc54c-240a-4075-8d55-2c47a9a9ecb4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:41.453451 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abcba9-5b8d-425a-aa1e-43b1ced13e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:41.454446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abcba9-5b8d-425a-aa1e-43b1ced13e5d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:41.454731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abcba9-5b8d-425a-aa1e-43b1ced13e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:41.455004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abcba9-5b8d-425a-aa1e-43b1ced13e5d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:41.455669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abcba9-5b8d-425a-aa1e-43b1ced13e5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:41.457308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abcba9-5b8d-425a-aa1e-43b1ced13e5d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f1abcba9-5b8d-425a-aa1e-43b1ced13e5d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:41.469745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:41.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:41.471745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 432/853] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jul 4 09:02:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:41.472658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984254e-b294-48a8-b23e-6f309a7a2653 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:41.472658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984254e-b294-48a8-b23e-6f309a7a2653 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:41.473440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984254e-b294-48a8-b23e-6f309a7a2653 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:41.474492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 433/854] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:41.477926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984254e-b294-48a8-b23e-6f309a7a2653 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:41.497988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190b26a-4dc4-4132-88d9-1f86712b04d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:41.498648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190b26a-4dc4-4132-88d9-1f86712b04d0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:41.498930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190b26a-4dc4-4132-88d9-1f86712b04d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:41.499205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190b26a-4dc4-4132-88d9-1f86712b04d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:41.500180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190b26a-4dc4-4132-88d9-1f86712b04d0 None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=8cbb65dbfadf4c63a272f5318efa875a) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:41.500493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190b26a-4dc4-4132-88d9-1f86712b04d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:41.502014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190b26a-4dc4-4132-88d9-1f86712b04d0 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:41.510047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 422/855] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:41 2022] GET /identity/v3/role_assignments?user.id=8cbb65dbfadf4c63a272f5318efa875a&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=2302517080ac42578aa81c606dd0ff2f => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:41.510047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801811f-fce6-480a-98e2-8799c995b7cc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:41.510047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801811f-fce6-480a-98e2-8799c995b7cc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:41.510355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801811f-fce6-480a-98e2-8799c995b7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:41.536146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e801811f-fce6-480a-98e2-8799c995b7cc None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:41.537592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e801811f-fce6-480a-98e2-8799c995b7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:41.576888 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:41.606684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984254e-b294-48a8-b23e-6f309a7a2653 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:41.645938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 423/856] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:41.650734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75e462-86e3-4c04-8be8-1813d135a2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:41.651435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75e462-86e3-4c04-8be8-1813d135a2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:41.651724 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75e462-86e3-4c04-8be8-1813d135a2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:41.652001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75e462-86e3-4c04-8be8-1813d135a2a8 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:41.652745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75e462-86e3-4c04-8be8-1813d135a2a8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:41.653062 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75e462-86e3-4c04-8be8-1813d135a2a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:41.654554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75e462-86e3-4c04-8be8-1813d135a2a8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:41.660795 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 434/857] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 4 09:02:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:41.670959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801811f-fce6-480a-98e2-8799c995b7cc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:41.671307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75e462-86e3-4c04-8be8-1813d135a2a8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:41.706131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:41.787407 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801811f-fce6-480a-98e2-8799c995b7cc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:41.795602 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75e462-86e3-4c04-8be8-1813d135a2a8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:41.822108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacfc16-2b9e-4f25-aeef-67bca66f6e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:41.822805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacfc16-2b9e-4f25-aeef-67bca66f6e32 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:41.823055 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacfc16-2b9e-4f25-aeef-67bca66f6e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:41.823334 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacfc16-2b9e-4f25-aeef-67bca66f6e32 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:41.824047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacfc16-2b9e-4f25-aeef-67bca66f6e32 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:41.824357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacfc16-2b9e-4f25-aeef-67bca66f6e32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:41.825737 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacfc16-2b9e-4f25-aeef-67bca66f6e32 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:41.830962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 424/858] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 4 09:02:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:41.831088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce211f5-7d8c-44d7-a0cb-f3a31d6336b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:41.832091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce211f5-7d8c-44d7-a0cb-f3a31d6336b1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:41.832305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce211f5-7d8c-44d7-a0cb-f3a31d6336b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:41.832721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce211f5-7d8c-44d7-a0cb-f3a31d6336b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:41.848537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce211f5-7d8c-44d7-a0cb-f3a31d6336b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-5ce211f5-7d8c-44d7-a0cb-f3a31d6336b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:41.859099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:41.860196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:41.862009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 435/859] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:02:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:41.867002 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacfc16-2b9e-4f25-aeef-67bca66f6e32 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:41.899720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:41.985994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacfc16-2b9e-4f25-aeef-67bca66f6e32 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:42.023068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f30b76-bab3-4429-9cbe-b81c28da1102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:42.024136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f30b76-bab3-4429-9cbe-b81c28da1102 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:42.024567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f30b76-bab3-4429-9cbe-b81c28da1102 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:42.024990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f30b76-bab3-4429-9cbe-b81c28da1102 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:42.025915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f30b76-bab3-4429-9cbe-b81c28da1102 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:42.026362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f30b76-bab3-4429-9cbe-b81c28da1102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:42.027869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f30b76-bab3-4429-9cbe-b81c28da1102 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:42.037504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 425/860] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:02:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:42.042818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce211f5-7d8c-44d7-a0cb-f3a31d6336b1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:42.134353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:42.151164 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce211f5-7d8c-44d7-a0cb-f3a31d6336b1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:42.185691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c38afe-de8c-4bf1-9883-04ff151fa148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:42.186524 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c38afe-de8c-4bf1-9883-04ff151fa148 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:42.186800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c38afe-de8c-4bf1-9883-04ff151fa148 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:42.187072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c38afe-de8c-4bf1-9883-04ff151fa148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:42.200473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c38afe-de8c-4bf1-9883-04ff151fa148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:42.201966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c38afe-de8c-4bf1-9883-04ff151fa148 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-04c38afe-de8c-4bf1-9883-04ff151fa148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:42.220847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:42.221813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:42.222797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:42.222797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:42.222797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:42.222797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:42.222797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 436/861] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:42.227771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f30b76-bab3-4429-9cbe-b81c28da1102 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:42.322697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:42.336280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f30b76-bab3-4429-9cbe-b81c28da1102 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:42.373375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a7c5f-cd30-4e50-9068-281ac5b57b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:42.374078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7c5f-cd30-4e50-9068-281ac5b57b34 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:42.374368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7c5f-cd30-4e50-9068-281ac5b57b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:42.374647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a7c5f-cd30-4e50-9068-281ac5b57b34 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:42.375387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7c5f-cd30-4e50-9068-281ac5b57b34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:42.375695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7c5f-cd30-4e50-9068-281ac5b57b34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:42.377068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a7c5f-cd30-4e50-9068-281ac5b57b34 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:42.391352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 426/862] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:42.397129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c38afe-de8c-4bf1-9883-04ff151fa148 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:42.500426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c38afe-de8c-4bf1-9883-04ff151fa148 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:42.536572 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8d468-da6d-4d74-a4c6-c01bf15f99a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:42.537395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8d468-da6d-4d74-a4c6-c01bf15f99a8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:42.537676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8d468-da6d-4d74-a4c6-c01bf15f99a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:42.537964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8d468-da6d-4d74-a4c6-c01bf15f99a8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:42.545670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8d468-da6d-4d74-a4c6-c01bf15f99a8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:42.547172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8d468-da6d-4d74-a4c6-c01bf15f99a8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:42.550408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-fbb8d468-da6d-4d74-a4c6-c01bf15f99a8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:42.563139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:42.565353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:42.567648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:42.567648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:42.567648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:42.567648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:42.567648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:42.567648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:42.567648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:42.567648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:42.567648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:42.567648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:42.567648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 437/863] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 4 09:02:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:42.573988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a7c5f-cd30-4e50-9068-281ac5b57b34 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:42.614206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591c0a-22bd-47d0-bf3d-7e967d362827 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:42.614206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591c0a-22bd-47d0-bf3d-7e967d362827 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:42.614206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591c0a-22bd-47d0-bf3d-7e967d362827 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:42.614910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 438/864] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:42.620094 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017ee02-738d-4f82-8154-510c29f87933 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:42.620578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017ee02-738d-4f82-8154-510c29f87933 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:42.621032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017ee02-738d-4f82-8154-510c29f87933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:42.638984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d017ee02-738d-4f82-8154-510c29f87933 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:42.640509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d017ee02-738d-4f82-8154-510c29f87933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:42.683793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a7c5f-cd30-4e50-9068-281ac5b57b34 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:42.708410 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 439/865] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:42.723723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752741c-efbb-4af4-bb51-789057db6552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:42.724478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752741c-efbb-4af4-bb51-789057db6552 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:42.724832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752741c-efbb-4af4-bb51-789057db6552 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:42.725179 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752741c-efbb-4af4-bb51-789057db6552 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:42.726131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752741c-efbb-4af4-bb51-789057db6552 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:42.726507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752741c-efbb-4af4-bb51-789057db6552 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:42.728003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752741c-efbb-4af4-bb51-789057db6552 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:42.734956 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017ee02-738d-4f82-8154-510c29f87933 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:42.747881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 427/866] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 4 09:02:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:42.754123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752741c-efbb-4af4-bb51-789057db6552 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:42.766280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:42.863737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017ee02-738d-4f82-8154-510c29f87933 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:42.875561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752741c-efbb-4af4-bb51-789057db6552 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:42.893738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:42.903518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935eceb-20e2-4a88-9d48-4715bbcb0482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:42.904276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935eceb-20e2-4a88-9d48-4715bbcb0482 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:42.904628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935eceb-20e2-4a88-9d48-4715bbcb0482 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:42.904969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935eceb-20e2-4a88-9d48-4715bbcb0482 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:42.905788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935eceb-20e2-4a88-9d48-4715bbcb0482 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:42.907275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935eceb-20e2-4a88-9d48-4715bbcb0482 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:42.923604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e45abc-b5b7-4c89-a7a3-ca05c04c3665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:42.924170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 440/867] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:02:42 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:42.925920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e45abc-b5b7-4c89-a7a3-ca05c04c3665 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:42.932181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e45abc-b5b7-4c89-a7a3-ca05c04c3665 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:42.932465 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e45abc-b5b7-4c89-a7a3-ca05c04c3665 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:43.025850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e45abc-b5b7-4c89-a7a3-ca05c04c3665 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:43.027803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e45abc-b5b7-4c89-a7a3-ca05c04c3665 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:43.037920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:43.142104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 428/868] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 09:02:43.147786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935eceb-20e2-4a88-9d48-4715bbcb0482 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:43.262556 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935eceb-20e2-4a88-9d48-4715bbcb0482 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:43.297112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de317b32-f4ce-462b-a447-8fda66ca18c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:43.298346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de317b32-f4ce-462b-a447-8fda66ca18c4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:43.298832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de317b32-f4ce-462b-a447-8fda66ca18c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:43.299302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de317b32-f4ce-462b-a447-8fda66ca18c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:43.312134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de317b32-f4ce-462b-a447-8fda66ca18c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:43.313907 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de317b32-f4ce-462b-a447-8fda66ca18c4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-de317b32-f4ce-462b-a447-8fda66ca18c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:43.331535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:43.332984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:43.333919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:43.333919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:43.333919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:43.333919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:43.335772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 441/869] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:43.341906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e45abc-b5b7-4c89-a7a3-ca05c04c3665 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:43.446147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e45abc-b5b7-4c89-a7a3-ca05c04c3665 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:43.453865 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:43.482445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143c869-c58a-4d0b-8934-a08487fa2a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:43.483113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143c869-c58a-4d0b-8934-a08487fa2a1a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:43.483398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143c869-c58a-4d0b-8934-a08487fa2a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:43.483675 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143c869-c58a-4d0b-8934-a08487fa2a1a None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:43.484400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143c869-c58a-4d0b-8934-a08487fa2a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:43.484709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143c869-c58a-4d0b-8934-a08487fa2a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:43.486097 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143c869-c58a-4d0b-8934-a08487fa2a1a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:43.498431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 429/870] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:43.504473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de317b32-f4ce-462b-a447-8fda66ca18c4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:43.540810 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:43.612187 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de317b32-f4ce-462b-a447-8fda66ca18c4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:43.647237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b584bd-5df4-4c22-9a28-0232ba8a6eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:43.648360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b584bd-5df4-4c22-9a28-0232ba8a6eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:43.648819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b584bd-5df4-4c22-9a28-0232ba8a6eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:43.649299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b584bd-5df4-4c22-9a28-0232ba8a6eb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:43.656774 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b584bd-5df4-4c22-9a28-0232ba8a6eb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:43.658510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b584bd-5df4-4c22-9a28-0232ba8a6eb4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c5b584bd-5df4-4c22-9a28-0232ba8a6eb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:43.673943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:43.674897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:43.675858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:43.675858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:43.675858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:43.675858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:43.675858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:43.675858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:43.675858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:43.675858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:43.678152 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 442/871] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:43.684123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89212a-7b4e-4073-9732-c4590a209906 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:43.684412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89212a-7b4e-4073-9732-c4590a209906 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:43.684688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89212a-7b4e-4073-9732-c4590a209906 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:43.686870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b584bd-5df4-4c22-9a28-0232ba8a6eb4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:43.687243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 430/872] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:43.691160 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3f817-c60a-4f64-a8a3-8791bfdc4907 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:43.691446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3f817-c60a-4f64-a8a3-8791bfdc4907 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:43.691715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3f817-c60a-4f64-a8a3-8791bfdc4907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:43.708684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-68b3f817-c60a-4f64-a8a3-8791bfdc4907 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:43.709787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-68b3f817-c60a-4f64-a8a3-8791bfdc4907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:43.789962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 431/873] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:02:43.802372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:43.805963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3f817-c60a-4f64-a8a3-8791bfdc4907 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:43.811948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b584bd-5df4-4c22-9a28-0232ba8a6eb4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:43.856693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f143d94-ae58-4e06-9969-6b720378fbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:43.857408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f143d94-ae58-4e06-9969-6b720378fbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:43.858096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f143d94-ae58-4e06-9969-6b720378fbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:43.858380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f143d94-ae58-4e06-9969-6b720378fbc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:43.859183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f143d94-ae58-4e06-9969-6b720378fbc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:43.859494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f143d94-ae58-4e06-9969-6b720378fbc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:43.860976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f143d94-ae58-4e06-9969-6b720378fbc6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:43.874185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 443/874] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:43.880143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f143d94-ae58-4e06-9969-6b720378fbc6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:43.922435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:43.930539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3f817-c60a-4f64-a8a3-8791bfdc4907 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:43.967195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8426d-1463-449a-98d1-0f47731b7931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:43.968346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8426d-1463-449a-98d1-0f47731b7931 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:43.969030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8426d-1463-449a-98d1-0f47731b7931 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:43.969509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8426d-1463-449a-98d1-0f47731b7931 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:43.970451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8426d-1463-449a-98d1-0f47731b7931 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:43.972018 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8426d-1463-449a-98d1-0f47731b7931 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:43.984695 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f143d94-ae58-4e06-9969-6b720378fbc6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-6ee8426d-1463-449a-98d1-0f47731b7931 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:43.986189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:43.987196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:43.988224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:43.989853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 432/875] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 4 09:02:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:43.991064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157ea09-c8d1-4aa5-bdd7-75ba38906f02 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:43.991357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157ea09-c8d1-4aa5-bdd7-75ba38906f02 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:43.991636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157ea09-c8d1-4aa5-bdd7-75ba38906f02 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:43.993008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 433/876] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:43.997788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157ea09-c8d1-4aa5-bdd7-75ba38906f02 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:44.026738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dc526-7a3d-45b0-9428-592502dbf03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:44.027399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dc526-7a3d-45b0-9428-592502dbf03d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:44.027683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dc526-7a3d-45b0-9428-592502dbf03d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:44.027958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dc526-7a3d-45b0-9428-592502dbf03d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:44.028796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533dc526-7a3d-45b0-9428-592502dbf03d None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=6bb6d63553794d10a8cba042f8560348) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:44.029112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533dc526-7a3d-45b0-9428-592502dbf03d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:44.030586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533dc526-7a3d-45b0-9428-592502dbf03d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:44.045913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 444/877] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:43 2022] GET /identity/v3/role_assignments?user.id=6bb6d63553794d10a8cba042f8560348&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:44.045913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767640cf-339a-49fd-87fb-1b9bd2083a58 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:44.045913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767640cf-339a-49fd-87fb-1b9bd2083a58 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:44.045913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767640cf-339a-49fd-87fb-1b9bd2083a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:44.068046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-767640cf-339a-49fd-87fb-1b9bd2083a58 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:44.069465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-767640cf-339a-49fd-87fb-1b9bd2083a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:44.089200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:44.121637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157ea09-c8d1-4aa5-bdd7-75ba38906f02 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:44.164101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 445/878] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:44 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:44.177986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fafb01-20a5-4d5a-af86-fc2893e9e94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:44.177986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fafb01-20a5-4d5a-af86-fc2893e9e94c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:44.177986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fafb01-20a5-4d5a-af86-fc2893e9e94c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:44.177986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fafb01-20a5-4d5a-af86-fc2893e9e94c None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:44.178435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fafb01-20a5-4d5a-af86-fc2893e9e94c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:44.178435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fafb01-20a5-4d5a-af86-fc2893e9e94c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:44.179424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fafb01-20a5-4d5a-af86-fc2893e9e94c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:44.188478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767640cf-339a-49fd-87fb-1b9bd2083a58 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:44.190069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 434/879] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 4 09:02:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:44.193436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fafb01-20a5-4d5a-af86-fc2893e9e94c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:44.227502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:44.319076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767640cf-339a-49fd-87fb-1b9bd2083a58 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:44.326858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fafb01-20a5-4d5a-af86-fc2893e9e94c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:44.364792 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23639d-2f0c-4eff-a265-9122072348b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:44.366324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23639d-2f0c-4eff-a265-9122072348b4 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:44.366904 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23639d-2f0c-4eff-a265-9122072348b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:44.367768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23639d-2f0c-4eff-a265-9122072348b4 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:44.369019 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23639d-2f0c-4eff-a265-9122072348b4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:44.371041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23639d-2f0c-4eff-a265-9122072348b4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:44.376051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5438aee-131f-46fd-9509-66796a98afdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:44.376793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5438aee-131f-46fd-9509-66796a98afdc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:44.377143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5438aee-131f-46fd-9509-66796a98afdc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:44.377746 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5438aee-131f-46fd-9509-66796a98afdc None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:44.378568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5438aee-131f-46fd-9509-66796a98afdc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:44.379507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5438aee-131f-46fd-9509-66796a98afdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:44.380971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5438aee-131f-46fd-9509-66796a98afdc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:44.386759 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 435/880] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 4 09:02:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:44.396957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 446/881] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:02:44 2022] POST /identity/v3/services => generated 269 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:45.128576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541950b-74c4-4ff5-8e9f-d46359465434 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:45.129146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541950b-74c4-4ff5-8e9f-d46359465434 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:45.129146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541950b-74c4-4ff5-8e9f-d46359465434 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:45.130802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 436/882] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:45.136360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5524f-9086-4067-8ba6-5967b4c8dd33 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:45.136781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5524f-9086-4067-8ba6-5967b4c8dd33 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:45.136952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5524f-9086-4067-8ba6-5967b4c8dd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:45.156152 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-aed5524f-9086-4067-8ba6-5967b4c8dd33 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:45.157351 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aed5524f-9086-4067-8ba6-5967b4c8dd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:45.159517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:45.233103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 447/883] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:45.256375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541950b-74c4-4ff5-8e9f-d46359465434 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:45.350916 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:45.365355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541950b-74c4-4ff5-8e9f-d46359465434 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:45.400707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bd743-42c3-423b-bc97-1c7bbe6e4dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:45.401491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bd743-42c3-423b-bc97-1c7bbe6e4dcc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:45.401775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bd743-42c3-423b-bc97-1c7bbe6e4dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:45.402132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bd743-42c3-423b-bc97-1c7bbe6e4dcc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:45.402795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bd743-42c3-423b-bc97-1c7bbe6e4dcc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:45.404281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bd743-42c3-423b-bc97-1c7bbe6e4dcc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c58bd743-42c3-423b-bc97-1c7bbe6e4dcc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:45.416207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:45.417199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:45.418106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 437/884] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:02:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:45.423666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5524f-9086-4067-8ba6-5967b4c8dd33 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:45.523004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5524f-9086-4067-8ba6-5967b4c8dd33 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:45.560179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaed8d1-4fc7-485d-889c-e3e7a0b16a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:45.561271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed8d1-4fc7-485d-889c-e3e7a0b16a7b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:45.561751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed8d1-4fc7-485d-889c-e3e7a0b16a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:45.562247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed8d1-4fc7-485d-889c-e3e7a0b16a7b None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:45.563151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaed8d1-4fc7-485d-889c-e3e7a0b16a7b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:45.563667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaed8d1-4fc7-485d-889c-e3e7a0b16a7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:45.565240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaed8d1-4fc7-485d-889c-e3e7a0b16a7b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:45.566693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:45.570961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 448/885] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:02:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:45.576655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bd743-42c3-423b-bc97-1c7bbe6e4dcc None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:45.682398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bd743-42c3-423b-bc97-1c7bbe6e4dcc None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:45.719214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b2f1c-a16e-497c-b317-7c747b80f2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:45.719888 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b2f1c-a16e-497c-b317-7c747b80f2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:45.720198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b2f1c-a16e-497c-b317-7c747b80f2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:45.720452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b2f1c-a16e-497c-b317-7c747b80f2d2 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:45.721172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b2f1c-a16e-497c-b317-7c747b80f2d2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:45.721481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b2f1c-a16e-497c-b317-7c747b80f2d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:45.723002 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b2f1c-a16e-497c-b317-7c747b80f2d2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:45.728286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 438/886] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:02:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:45.734157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaed8d1-4fc7-485d-889c-e3e7a0b16a7b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:45.840782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaed8d1-4fc7-485d-889c-e3e7a0b16a7b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:45.875401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7999c1-2c65-4113-97d0-aafb3edb71ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:45.876072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7999c1-2c65-4113-97d0-aafb3edb71ee None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:45.876355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7999c1-2c65-4113-97d0-aafb3edb71ee None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:45.876631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7999c1-2c65-4113-97d0-aafb3edb71ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:45.877753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7999c1-2c65-4113-97d0-aafb3edb71ee None admin] RBAC: Adding query filter params (interface=public, service_id=321063472958468e8c99ae247752f737, region_id=RegionOne) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:45.878097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7999c1-2c65-4113-97d0-aafb3edb71ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:45.879639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7999c1-2c65-4113-97d0-aafb3edb71ee None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:45.885293 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 449/887] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 4 09:02:45 2022] GET /identity/v3/endpoints?service_id=321063472958468e8c99ae247752f737&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:45.891033 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b2f1c-a16e-497c-b317-7c747b80f2d2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:45.911371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:46.001112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b2f1c-a16e-497c-b317-7c747b80f2d2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:46.036212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c93654-f37c-4f3c-9780-9d57deb184f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:46.036885 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c93654-f37c-4f3c-9780-9d57deb184f6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:46.037171 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c93654-f37c-4f3c-9780-9d57deb184f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:46.037452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c93654-f37c-4f3c-9780-9d57deb184f6 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:46.038172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c93654-f37c-4f3c-9780-9d57deb184f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:46.039554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c93654-f37c-4f3c-9780-9d57deb184f6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:46.046086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 439/888] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 4 09:02:45 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:02:46.302171 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f4da9-4b67-4f74-8523-cbe3f4bb9a21 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:46.302702 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f4da9-4b67-4f74-8523-cbe3f4bb9a21 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:46.302702 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f4da9-4b67-4f74-8523-cbe3f4bb9a21 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:46.304280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 450/889] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:46.309611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e3ca9-349a-4190-a15b-37f903047f8d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:46.310076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e3ca9-349a-4190-a15b-37f903047f8d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:46.310235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e3ca9-349a-4190-a15b-37f903047f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:46.327672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e00e3ca9-349a-4190-a15b-37f903047f8d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:46.328852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e00e3ca9-349a-4190-a15b-37f903047f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:46.397213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 440/890] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:02:46.420822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f4da9-4b67-4f74-8523-cbe3f4bb9a21 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:46.473310 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea0a7b-c8b9-4d5a-8710-0e3fad621885 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:46.474010 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea0a7b-c8b9-4d5a-8710-0e3fad621885 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:46.474483 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea0a7b-c8b9-4d5a-8710-0e3fad621885 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:46.477283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 441/891] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:46.481562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4181f4-f458-4c14-a200-5b05ba64e85d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:46.482114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4181f4-f458-4c14-a200-5b05ba64e85d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:46.482606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4181f4-f458-4c14-a200-5b05ba64e85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:46.486893 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:46.502269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ac4181f4-f458-4c14-a200-5b05ba64e85d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:46.504294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ac4181f4-f458-4c14-a200-5b05ba64e85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:46.531559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f4da9-4b67-4f74-8523-cbe3f4bb9a21 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:46.531797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:46.572188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb474126-89e5-43d2-8f8c-e1177fa8184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:46.572924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474126-89e5-43d2-8f8c-e1177fa8184b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:46.573206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474126-89e5-43d2-8f8c-e1177fa8184b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:46.573484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb474126-89e5-43d2-8f8c-e1177fa8184b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:46.587068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb474126-89e5-43d2-8f8c-e1177fa8184b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:46.592008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 442/892] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:02:46.593877 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ae7f-aa87-4143-9e73-751cd552d548 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:46.594482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ae7f-aa87-4143-9e73-751cd552d548 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:46.594933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983ae7f-aa87-4143-9e73-751cd552d548 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-eb474126-89e5-43d2-8f8c-e1177fa8184b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:46.595794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:46.596751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:02:46.597640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:46.599377 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 443/894] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:46.602160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 451/894] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 4 09:02:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:46.605488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599dc7e-89b0-4942-8662-62cee971a919 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:46.605981 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599dc7e-89b0-4942-8662-62cee971a919 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:46.606463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599dc7e-89b0-4942-8662-62cee971a919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:46.606828 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ae7f-aa87-4143-9e73-751cd552d548 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:46.627908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3599dc7e-89b0-4942-8662-62cee971a919 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:46.629093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3599dc7e-89b0-4942-8662-62cee971a919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:46.697932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 452/895] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:46.698603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599dc7e-89b0-4942-8662-62cee971a919 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:46.728328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983ae7f-aa87-4143-9e73-751cd552d548 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:46.758590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:46.764054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d00fcc-0a07-4dc0-a2a8-f53e66ac7410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:46.764726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d00fcc-0a07-4dc0-a2a8-f53e66ac7410 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:46.765011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d00fcc-0a07-4dc0-a2a8-f53e66ac7410 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:46.765286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d00fcc-0a07-4dc0-a2a8-f53e66ac7410 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:46.766067 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d00fcc-0a07-4dc0-a2a8-f53e66ac7410 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:46.766375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d00fcc-0a07-4dc0-a2a8-f53e66ac7410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:46.767742 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d00fcc-0a07-4dc0-a2a8-f53e66ac7410 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:46.781180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 444/896] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 4 09:02:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:46.781180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d00fcc-0a07-4dc0-a2a8-f53e66ac7410 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:46.784210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:46.814269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599dc7e-89b0-4942-8662-62cee971a919 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:46.848897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6ed78-00fe-454e-a1dd-513e9babf2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:46.849742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6ed78-00fe-454e-a1dd-513e9babf2be None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:46.850060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6ed78-00fe-454e-a1dd-513e9babf2be None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:46.850297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6ed78-00fe-454e-a1dd-513e9babf2be None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:46.850960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6ed78-00fe-454e-a1dd-513e9babf2be None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:46.852502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6ed78-00fe-454e-a1dd-513e9babf2be None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-6cd6ed78-00fe-454e-a1dd-513e9babf2be None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:46.865531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:46.866758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:46.867616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 453/897] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jul 4 09:02:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:46.868474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6ed78-00fe-454e-a1dd-513e9babf2be None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:46.892232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d00fcc-0a07-4dc0-a2a8-f53e66ac7410 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:46.928656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25881306-05e6-48d6-b038-61bd82c0c056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:46.929316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25881306-05e6-48d6-b038-61bd82c0c056 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:46.929600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25881306-05e6-48d6-b038-61bd82c0c056 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:46.929873 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25881306-05e6-48d6-b038-61bd82c0c056 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:46.930617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25881306-05e6-48d6-b038-61bd82c0c056 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:46.931991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25881306-05e6-48d6-b038-61bd82c0c056 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:46.946749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 445/898] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:02:46 2022] POST /identity/v3/services => generated 241 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:46.947447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25881306-05e6-48d6-b038-61bd82c0c056 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:46.995633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6ed78-00fe-454e-a1dd-513e9babf2be None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:47.024823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:47.026509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:47.063675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9ff56-918b-428a-9cda-c124ed2f40f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:47.064900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9ff56-918b-428a-9cda-c124ed2f40f2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:47.066210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9ff56-918b-428a-9cda-c124ed2f40f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:47.066711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9ff56-918b-428a-9cda-c124ed2f40f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:47.079113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25881306-05e6-48d6-b038-61bd82c0c056 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:47.084916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9ff56-918b-428a-9cda-c124ed2f40f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:47.086790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9ff56-918b-428a-9cda-c124ed2f40f2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-81a9ff56-918b-428a-9cda-c124ed2f40f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:47.110835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:47.111793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:47.112706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:47.112706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:47.112706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:47.112706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:47.112706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 454/899] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:47.118540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9ff56-918b-428a-9cda-c124ed2f40f2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:47.126896 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb279db-75c0-4d0c-a34f-82566cfb4b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:47.127894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb279db-75c0-4d0c-a34f-82566cfb4b9f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:47.128176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb279db-75c0-4d0c-a34f-82566cfb4b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:47.128446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb279db-75c0-4d0c-a34f-82566cfb4b9f None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:47.131457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb279db-75c0-4d0c-a34f-82566cfb4b9f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:47.131775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb279db-75c0-4d0c-a34f-82566cfb4b9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:47.133177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb279db-75c0-4d0c-a34f-82566cfb4b9f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:47.142034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 446/900] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 4 09:02:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:47.144361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb279db-75c0-4d0c-a34f-82566cfb4b9f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:47.241220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9ff56-918b-428a-9cda-c124ed2f40f2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:47.255087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb279db-75c0-4d0c-a34f-82566cfb4b9f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:47.288062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:47.288257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a03d06-cee5-4f6b-9417-86f44bfc265e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:47.289121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a03d06-cee5-4f6b-9417-86f44bfc265e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:47.289121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a03d06-cee5-4f6b-9417-86f44bfc265e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:47.289379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a03d06-cee5-4f6b-9417-86f44bfc265e None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:47.290302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a03d06-cee5-4f6b-9417-86f44bfc265e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:47.290611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a03d06-cee5-4f6b-9417-86f44bfc265e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:47.292722 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a03d06-cee5-4f6b-9417-86f44bfc265e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:47.300952 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb4ee3-aa68-4729-a229-2de2f3b50623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:47.302533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb4ee3-aa68-4729-a229-2de2f3b50623 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:47.303028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb4ee3-aa68-4729-a229-2de2f3b50623 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:47.304017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb4ee3-aa68-4729-a229-2de2f3b50623 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:47.305260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb4ee3-aa68-4729-a229-2de2f3b50623 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:47.305776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb4ee3-aa68-4729-a229-2de2f3b50623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:47.307429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb4ee3-aa68-4729-a229-2de2f3b50623 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:47.308255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 455/901] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:47.313352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a03d06-cee5-4f6b-9417-86f44bfc265e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:47.326914 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 447/902] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 4 09:02:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:47.332758 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb4ee3-aa68-4729-a229-2de2f3b50623 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:47.344182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:47.442473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a03d06-cee5-4f6b-9417-86f44bfc265e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:47.458559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb4ee3-aa68-4729-a229-2de2f3b50623 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:47.487687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24384355-6cfb-4a73-a18f-d0336277cba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:47.489511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384355-6cfb-4a73-a18f-d0336277cba2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:47.489993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384355-6cfb-4a73-a18f-d0336277cba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:47.490459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384355-6cfb-4a73-a18f-d0336277cba2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:47.500133 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355abc6-9360-460d-99fe-5d4a570c17bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:47.500556 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24384355-6cfb-4a73-a18f-d0336277cba2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:47.501001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355abc6-9360-460d-99fe-5d4a570c17bc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:47.501374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355abc6-9360-460d-99fe-5d4a570c17bc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:47.501740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355abc6-9360-460d-99fe-5d4a570c17bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:47.502959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355abc6-9360-460d-99fe-5d4a570c17bc None admin] RBAC: Adding query filter params (interface=public, service_id=beac68d005694d88939d219bf241c626, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:47.503320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24384355-6cfb-4a73-a18f-d0336277cba2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:47.503442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355abc6-9360-460d-99fe-5d4a570c17bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:47.504966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355abc6-9360-460d-99fe-5d4a570c17bc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:47.514054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 448/903] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 4 09:02:47 2022] GET /identity/v3/endpoints?service_id=beac68d005694d88939d219bf241c626&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:47.518540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355abc6-9360-460d-99fe-5d4a570c17bc None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-24384355-6cfb-4a73-a18f-d0336277cba2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:47.527928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:47.529045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:47.530336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:47.530336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:47.530336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:47.530336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:47.530336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:47.530336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:47.530336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:47.530336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:47.530336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:47.530336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:47.534536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 456/904] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 4 09:02:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:47.535892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24384355-6cfb-4a73-a18f-d0336277cba2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:47.588100 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:47.625354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:47.641102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355abc6-9360-460d-99fe-5d4a570c17bc None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:47.644605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24384355-6cfb-4a73-a18f-d0336277cba2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:47.678927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33639f0b-096d-4f79-9ea9-cee2ff5e083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:47.680079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33639f0b-096d-4f79-9ea9-cee2ff5e083d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:47.680628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33639f0b-096d-4f79-9ea9-cee2ff5e083d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:47.681303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33639f0b-096d-4f79-9ea9-cee2ff5e083d None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:47.682544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33639f0b-096d-4f79-9ea9-cee2ff5e083d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:47.684310 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33639f0b-096d-4f79-9ea9-cee2ff5e083d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:47.692807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 449/905] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 4 09:02:47 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:02:47.704466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe45992-c602-4308-867f-4afb71b8b1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:47.705562 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe45992-c602-4308-867f-4afb71b8b1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:47.706265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe45992-c602-4308-867f-4afb71b8b1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:47.706790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe45992-c602-4308-867f-4afb71b8b1f0 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:47.707949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe45992-c602-4308-867f-4afb71b8b1f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:47.708506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe45992-c602-4308-867f-4afb71b8b1f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:47.710284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe45992-c602-4308-867f-4afb71b8b1f0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:47.732060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 457/906] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 4 09:02:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:47.741919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33639f0b-096d-4f79-9ea9-cee2ff5e083d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:47.851198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33639f0b-096d-4f79-9ea9-cee2ff5e083d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:47.870714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:47.888545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436e5bc-27bf-4a33-8a5f-8c92e335ba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:47.889204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436e5bc-27bf-4a33-8a5f-8c92e335ba9c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:47.889491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436e5bc-27bf-4a33-8a5f-8c92e335ba9c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:47.889769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436e5bc-27bf-4a33-8a5f-8c92e335ba9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:47.963041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436e5bc-27bf-4a33-8a5f-8c92e335ba9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:47.964441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436e5bc-27bf-4a33-8a5f-8c92e335ba9c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:48.074663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 450/907] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:47 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:02:48.080322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe45992-c602-4308-867f-4afb71b8b1f0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:48.164962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5cc05-72ac-47b0-a39f-30017883e263 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:48.165261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5cc05-72ac-47b0-a39f-30017883e263 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:48.165515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5cc05-72ac-47b0-a39f-30017883e263 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:48.167608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 451/908] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:48.175440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663529d-1bbc-4611-accb-8088c7889e0d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:48.175845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663529d-1bbc-4611-accb-8088c7889e0d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:48.176313 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663529d-1bbc-4611-accb-8088c7889e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:48.194214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe45992-c602-4308-867f-4afb71b8b1f0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:48.197656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d663529d-1bbc-4611-accb-8088c7889e0d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:48.198974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d663529d-1bbc-4611-accb-8088c7889e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:48.215649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:48.233952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ec6f2-55ec-4307-8519-0e5b4738edfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:48.234759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ec6f2-55ec-4307-8519-0e5b4738edfd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:48.235161 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ec6f2-55ec-4307-8519-0e5b4738edfd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:48.235437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ec6f2-55ec-4307-8519-0e5b4738edfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:48.246319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ec6f2-55ec-4307-8519-0e5b4738edfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:48.247763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ec6f2-55ec-4307-8519-0e5b4738edfd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:48.249638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a16ec6f2-55ec-4307-8519-0e5b4738edfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:48.269040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:48.270107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:48.271503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:48.271503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:48.271503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:48.271503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:48.275275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 458/909] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:48.276230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ec6f2-55ec-4307-8519-0e5b4738edfd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:48.297997 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 452/910] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:02:48.317311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663529d-1bbc-4611-accb-8088c7889e0d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:48.414124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ec6f2-55ec-4307-8519-0e5b4738edfd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:48.423120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663529d-1bbc-4611-accb-8088c7889e0d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:48.456106 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699d45d-0913-49ae-b65b-8c5f777aa2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:48.457182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699d45d-0913-49ae-b65b-8c5f777aa2ed None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:48.457715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699d45d-0913-49ae-b65b-8c5f777aa2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:48.458037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699d45d-0913-49ae-b65b-8c5f777aa2ed None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:48.458770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699d45d-0913-49ae-b65b-8c5f777aa2ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:48.459078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699d45d-0913-49ae-b65b-8c5f777aa2ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:48.460456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699d45d-0913-49ae-b65b-8c5f777aa2ed None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:48.462071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8957aee-bf6e-411a-b61c-947b21bf2095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:48.463089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8957aee-bf6e-411a-b61c-947b21bf2095 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:48.463469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8957aee-bf6e-411a-b61c-947b21bf2095 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:48.463844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8957aee-bf6e-411a-b61c-947b21bf2095 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:48.464606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8957aee-bf6e-411a-b61c-947b21bf2095 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:48.466153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8957aee-bf6e-411a-b61c-947b21bf2095 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:48.474262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 459/911] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:48.478672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699d45d-0913-49ae-b65b-8c5f777aa2ed None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-b8957aee-bf6e-411a-b61c-947b21bf2095 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:48.478993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:48.480146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 04 09:02:48.481267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:48.482336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 453/912] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:02:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:48.489084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8957aee-bf6e-411a-b61c-947b21bf2095 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:48.509074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:48.550015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:48.597495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699d45d-0913-49ae-b65b-8c5f777aa2ed None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:48.605865 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8957aee-bf6e-411a-b61c-947b21bf2095 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:48.632551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4381aa2-6433-4979-b29a-5ec06788fbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:48.633306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4381aa2-6433-4979-b29a-5ec06788fbca None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:48.633686 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4381aa2-6433-4979-b29a-5ec06788fbca None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:48.633990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4381aa2-6433-4979-b29a-5ec06788fbca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:48.640956 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4381aa2-6433-4979-b29a-5ec06788fbca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:48.642763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4381aa2-6433-4979-b29a-5ec06788fbca None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:48.644044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4c17c-6898-479e-ade7-9be68359d5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:48.645094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4c17c-6898-479e-ade7-9be68359d5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:48.645585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4c17c-6898-479e-ade7-9be68359d5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:48.646098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4c17c-6898-479e-ade7-9be68359d5e3 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:48.647015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4c17c-6898-479e-ade7-9be68359d5e3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:48.647556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4c17c-6898-479e-ade7-9be68359d5e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:48.649178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4c17c-6898-479e-ade7-9be68359d5e3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:48.654988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 454/913] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:02:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a4381aa2-6433-4979-b29a-5ec06788fbca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:48.657074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:48.658021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:48.658947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:48.658947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:48.658947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:48.658947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:48.658947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:48.658947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:48.658947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:48.658947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:48.662599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 460/914] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:48.667889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4381aa2-6433-4979-b29a-5ec06788fbca None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:48.669058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4c17c-6898-479e-ade7-9be68359d5e3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:48.750146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:48.774194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4381aa2-6433-4979-b29a-5ec06788fbca None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:48.775492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4c17c-6898-479e-ade7-9be68359d5e3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:48.789523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:48.811745 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea427c6-164d-4251-a8df-876bdca7b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:48.812418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea427c6-164d-4251-a8df-876bdca7b9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:48.812706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea427c6-164d-4251-a8df-876bdca7b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:48.812996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea427c6-164d-4251-a8df-876bdca7b9f4 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:48.813460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da3d97-bcaf-471b-9344-90a351bfbab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:48.813716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea427c6-164d-4251-a8df-876bdca7b9f4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:48.814075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea427c6-164d-4251-a8df-876bdca7b9f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:48.814759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da3d97-bcaf-471b-9344-90a351bfbab4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:48.815227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da3d97-bcaf-471b-9344-90a351bfbab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:48.815447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea427c6-164d-4251-a8df-876bdca7b9f4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:48.815908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da3d97-bcaf-471b-9344-90a351bfbab4 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:48.817198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da3d97-bcaf-471b-9344-90a351bfbab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:48.817739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da3d97-bcaf-471b-9344-90a351bfbab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:48.819513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da3d97-bcaf-471b-9344-90a351bfbab4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:48.825501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 455/915] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 4 09:02:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:48.826488 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea427c6-164d-4251-a8df-876bdca7b9f4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:48.840391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 461/916] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:48.846356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da3d97-bcaf-471b-9344-90a351bfbab4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:48.940754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea427c6-164d-4251-a8df-876bdca7b9f4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:48.961331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da3d97-bcaf-471b-9344-90a351bfbab4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:48.975857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a04e43-08f4-4d51-b2a5-c2d8bccc836c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:48.976526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a04e43-08f4-4d51-b2a5-c2d8bccc836c None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:48.976813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a04e43-08f4-4d51-b2a5-c2d8bccc836c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:48.977095 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a04e43-08f4-4d51-b2a5-c2d8bccc836c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:48.977835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a04e43-08f4-4d51-b2a5-c2d8bccc836c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:48.979350 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a04e43-08f4-4d51-b2a5-c2d8bccc836c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:48.987693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:49.006013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 456/917] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 4 09:02:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:02:49.021199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e33b5-3e95-46cf-93ab-ef1068301662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:49.022392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e33b5-3e95-46cf-93ab-ef1068301662 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:49.022955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e33b5-3e95-46cf-93ab-ef1068301662 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:49.023475 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e33b5-3e95-46cf-93ab-ef1068301662 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:49.024548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e33b5-3e95-46cf-93ab-ef1068301662 None admin] RBAC: Adding query filter params (role.id=a455148dcf0d4182bf84438cb30422c9, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=6bb6d63553794d10a8cba042f8560348) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:49.025117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e33b5-3e95-46cf-93ab-ef1068301662 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:49.026967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e33b5-3e95-46cf-93ab-ef1068301662 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:49.033990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34309c5f-0e3e-4a2a-b921-d552d8cdc38b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:49.034951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34309c5f-0e3e-4a2a-b921-d552d8cdc38b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:49.035703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34309c5f-0e3e-4a2a-b921-d552d8cdc38b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:49.038646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 457/918] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:49.046619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3574b7-06a4-4d5e-8e17-41cc824c9a30 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:49.047167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3574b7-06a4-4d5e-8e17-41cc824c9a30 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:49.047344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 462/919] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:48 2022] GET /identity/v3/role_assignments?user.id=6bb6d63553794d10a8cba042f8560348&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=a455148dcf0d4182bf84438cb30422c9 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:49.054559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3574b7-06a4-4d5e-8e17-41cc824c9a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:49.076152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ec3574b7-06a4-4d5e-8e17-41cc824c9a30 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:49.077701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ec3574b7-06a4-4d5e-8e17-41cc824c9a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:49.165139 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 458/920] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:49.190041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e33b5-3e95-46cf-93ab-ef1068301662 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:49.232530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:49.315782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e33b5-3e95-46cf-93ab-ef1068301662 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:49.353892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e2b44-fe6d-45e5-8f33-5e34495fe155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:49.354720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e2b44-fe6d-45e5-8f33-5e34495fe155 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:49.355052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e2b44-fe6d-45e5-8f33-5e34495fe155 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:49.355331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e2b44-fe6d-45e5-8f33-5e34495fe155 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:49.355989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e2b44-fe6d-45e5-8f33-5e34495fe155 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:49.357363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e2b44-fe6d-45e5-8f33-5e34495fe155 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e88e2b44-fe6d-45e5-8f33-5e34495fe155 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:49.370125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:49.371358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:49.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 463/921] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 4 09:02:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:49.377411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3574b7-06a4-4d5e-8e17-41cc824c9a30 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:49.485029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3574b7-06a4-4d5e-8e17-41cc824c9a30 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:49.512931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:49.523042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004383b9-f16e-4ca1-849c-2a17a17c7c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:49.523929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004383b9-f16e-4ca1-849c-2a17a17c7c83 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:49.524353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004383b9-f16e-4ca1-849c-2a17a17c7c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:49.524794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004383b9-f16e-4ca1-849c-2a17a17c7c83 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:49.525641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004383b9-f16e-4ca1-849c-2a17a17c7c83 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:49.526125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004383b9-f16e-4ca1-849c-2a17a17c7c83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:49.527633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004383b9-f16e-4ca1-849c-2a17a17c7c83 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:49.533224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 459/922] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 4 09:02:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:49.539385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e2b44-fe6d-45e5-8f33-5e34495fe155 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:49.648332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:49.652083 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e2b44-fe6d-45e5-8f33-5e34495fe155 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:49.688077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdda9d-eaaa-4fa7-857f-97a87a8c2ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:49.688759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdda9d-eaaa-4fa7-857f-97a87a8c2ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:49.689048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdda9d-eaaa-4fa7-857f-97a87a8c2ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:49.689324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdda9d-eaaa-4fa7-857f-97a87a8c2ac3 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:49.690084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdda9d-eaaa-4fa7-857f-97a87a8c2ac3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:49.690405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdda9d-eaaa-4fa7-857f-97a87a8c2ac3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:49.692554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdda9d-eaaa-4fa7-857f-97a87a8c2ac3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:49.698590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 464/923] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 4 09:02:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:49.704431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004383b9-f16e-4ca1-849c-2a17a17c7c83 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:49.811933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004383b9-f16e-4ca1-849c-2a17a17c7c83 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:49.846190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88055feb-240e-4264-aeac-0ac7a4698ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:49.846854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88055feb-240e-4264-aeac-0ac7a4698ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:49.847135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88055feb-240e-4264-aeac-0ac7a4698ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:49.847457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88055feb-240e-4264-aeac-0ac7a4698ac7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:49.848234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88055feb-240e-4264-aeac-0ac7a4698ac7 None admin] RBAC: Adding query filter params (interface=public, service_id=6f7cd26a348843d09d25a1283276f771, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:49.848539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88055feb-240e-4264-aeac-0ac7a4698ac7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:49.850167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88055feb-240e-4264-aeac-0ac7a4698ac7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:49.855962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 460/924] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 4 09:02:49 2022] GET /identity/v3/endpoints?service_id=6f7cd26a348843d09d25a1283276f771&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:49.861243 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdda9d-eaaa-4fa7-857f-97a87a8c2ac3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:49.870546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8f23-1f25-46c5-b22e-73593a14198e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:49.870860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8f23-1f25-46c5-b22e-73593a14198e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:49.871115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8f23-1f25-46c5-b22e-73593a14198e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:49.872737 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 461/925] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:49.877916 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb7c81-9fd6-43bf-b981-0b7a266275ac None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:49.878256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb7c81-9fd6-43bf-b981-0b7a266275ac None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:49.878614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb7c81-9fd6-43bf-b981-0b7a266275ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:49.895336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e2fb7c81-9fd6-43bf-b981-0b7a266275ac None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:49.896457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e2fb7c81-9fd6-43bf-b981-0b7a266275ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:49.962729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:49.968976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdda9d-eaaa-4fa7-857f-97a87a8c2ac3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:49.970027 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 462/926] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:02:49.988359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb7c81-9fd6-43bf-b981-0b7a266275ac None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:50.007296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb743c5c-eba2-45f3-b372-6890078e2741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:50.008246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb743c5c-eba2-45f3-b372-6890078e2741 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:50.008539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb743c5c-eba2-45f3-b372-6890078e2741 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:50.008821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb743c5c-eba2-45f3-b372-6890078e2741 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:50.009513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb743c5c-eba2-45f3-b372-6890078e2741 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:50.011363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb743c5c-eba2-45f3-b372-6890078e2741 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:50.020852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 465/927] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 4 09:02:49 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:02:50.097022 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb7c81-9fd6-43bf-b981-0b7a266275ac None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:50.139699 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff22ae-2c9d-4f07-8476-dcd6a0dad5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:50.140551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff22ae-2c9d-4f07-8476-dcd6a0dad5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:50.140838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff22ae-2c9d-4f07-8476-dcd6a0dad5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:50.141123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff22ae-2c9d-4f07-8476-dcd6a0dad5f1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:50.141800 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff22ae-2c9d-4f07-8476-dcd6a0dad5f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:50.143241 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff22ae-2c9d-4f07-8476-dcd6a0dad5f1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-d0ff22ae-2c9d-4f07-8476-dcd6a0dad5f1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:50.156848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:50.157812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:50.158724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 463/928] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jul 4 09:02:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:50.164754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb743c5c-eba2-45f3-b372-6890078e2741 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:50.197723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:50.276490 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb743c5c-eba2-45f3-b372-6890078e2741 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:50.313847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f083a-2e69-477c-8089-37b9f1719fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:50.314896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f083a-2e69-477c-8089-37b9f1719fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:50.315360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f083a-2e69-477c-8089-37b9f1719fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:50.315815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f083a-2e69-477c-8089-37b9f1719fd5 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:50.316733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f083a-2e69-477c-8089-37b9f1719fd5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:50.317281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f083a-2e69-477c-8089-37b9f1719fd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:50.318864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f083a-2e69-477c-8089-37b9f1719fd5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:50.324511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 466/929] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 4 09:02:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:50.330388 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff22ae-2c9d-4f07-8476-dcd6a0dad5f1 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:50.382447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:50.435839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff22ae-2c9d-4f07-8476-dcd6a0dad5f1 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:50.470051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24178a4a-d91e-48ae-825c-8d9f70a6fa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:50.470770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24178a4a-d91e-48ae-825c-8d9f70a6fa25 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:50.471058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24178a4a-d91e-48ae-825c-8d9f70a6fa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:50.471339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24178a4a-d91e-48ae-825c-8d9f70a6fa25 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:50.472104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24178a4a-d91e-48ae-825c-8d9f70a6fa25 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:50.472583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24178a4a-d91e-48ae-825c-8d9f70a6fa25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:50.474081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24178a4a-d91e-48ae-825c-8d9f70a6fa25 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:50.479466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 464/930] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 4 09:02:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:50.485056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f083a-2e69-477c-8089-37b9f1719fd5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:50.577364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:50.587913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f083a-2e69-477c-8089-37b9f1719fd5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:50.622393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c026f-cfd0-4466-b40a-c3abf6b6f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:50.623062 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c026f-cfd0-4466-b40a-c3abf6b6f4d8 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:50.623348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c026f-cfd0-4466-b40a-c3abf6b6f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:50.623627 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c026f-cfd0-4466-b40a-c3abf6b6f4d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:50.624392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c026f-cfd0-4466-b40a-c3abf6b6f4d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:50.625787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c026f-cfd0-4466-b40a-c3abf6b6f4d8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:50.650103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 467/931] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 4 09:02:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:51.229259 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7b503-9c48-4f82-a595-2a17b8e1163d None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:51.230271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7b503-9c48-4f82-a595-2a17b8e1163d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:51.230775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7b503-9c48-4f82-a595-2a17b8e1163d None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:51.232463 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 465/932] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:51.234273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378754-ed61-4317-9b1e-fd433e2b89dc None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:51.234771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378754-ed61-4317-9b1e-fd433e2b89dc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:51.235247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378754-ed61-4317-9b1e-fd433e2b89dc None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:51.236857 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 468/933] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:51.241338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038e6c5-2c01-40e5-a54b-83c1a362027d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:51.241338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038e6c5-2c01-40e5-a54b-83c1a362027d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:51.241338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038e6c5-2c01-40e5-a54b-83c1a362027d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:51.243122 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56e9c-c266-44bb-9129-f88951381472 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:51.243746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56e9c-c266-44bb-9129-f88951381472 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:51.244257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56e9c-c266-44bb-9129-f88951381472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:51.255893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5038e6c5-2c01-40e5-a54b-83c1a362027d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:51.257083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5038e6c5-2c01-40e5-a54b-83c1a362027d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:51.263089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3da56e9c-c266-44bb-9129-f88951381472 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:51.264472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3da56e9c-c266-44bb-9129-f88951381472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:51.343945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 466/934] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:02:51.352926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 469/935] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:51.366810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038e6c5-2c01-40e5-a54b-83c1a362027d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:51.378731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da56e9c-c266-44bb-9129-f88951381472 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:51.408446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:51.471333 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:51.471803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038e6c5-2c01-40e5-a54b-83c1a362027d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:51.511908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346a4d3-58ea-42c0-bd7a-50a282208e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:51.512729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346a4d3-58ea-42c0-bd7a-50a282208e7f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:51.513015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346a4d3-58ea-42c0-bd7a-50a282208e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:51.513344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346a4d3-58ea-42c0-bd7a-50a282208e7f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:51.513743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da56e9c-c266-44bb-9129-f88951381472 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:51.526731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346a4d3-58ea-42c0-bd7a-50a282208e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-a346a4d3-58ea-42c0-bd7a-50a282208e7f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:51.536136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:51.537120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:02:51.538498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:51.541142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 467/936] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 4 09:02:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:51.551464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346a4d3-58ea-42c0-bd7a-50a282208e7f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:51.554410 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470f6dd-2ace-4b3b-b00d-a3fc35c08b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:51.555170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470f6dd-2ace-4b3b-b00d-a3fc35c08b05 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:51.556354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470f6dd-2ace-4b3b-b00d-a3fc35c08b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:51.556354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470f6dd-2ace-4b3b-b00d-a3fc35c08b05 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:51.556646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470f6dd-2ace-4b3b-b00d-a3fc35c08b05 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:51.558186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470f6dd-2ace-4b3b-b00d-a3fc35c08b05 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:51.559651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: WARNING keystone.server.flask.common [None req-b470f6dd-2ace-4b3b-b00d-a3fc35c08b05 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 04 09:02:51.579024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.sql.core [None req-b470f6dd-2ace-4b3b-b00d-a3fc35c08b05 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 09:02:51.579024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 09:02:51.579024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [parameters: {'id': '211451b2aaa24c6b980591883cef5ef0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 09:02:51.579024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80284) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b470f6dd-2ace-4b3b-b00d-a3fc35c08b05 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 211451b2aaa24c6b980591883cef5ef0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 211451b2aaa24c6b980591883cef5ef0. Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:02:51.587949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 04 09:02:51.589013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application yield resource Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise self.value Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application session.commit() Jul 04 09:02:51.590082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.flush() Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application compat.raise_( Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 09:02:51.590986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 09:02:51.591923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:02:51.592726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [parameters: {'id': '211451b2aaa24c6b980591883cef5ef0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:51.593601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:51.595767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:51.597467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:51.597467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 04 09:02:51.597467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 04 09:02:51.597467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:51.597467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:51.597467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 04 09:02:51.597467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 04 09:02:51.597467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 211451b2aaa24c6b980591883cef5ef0. Jul 04 09:02:51.597467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:51.597467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 470/937] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Jul 4 09:02:51 2022] POST /identity/v3/projects => generated 262 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:51.599377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470f6dd-2ace-4b3b-b00d-a3fc35c08b05 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:51.690858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:51.696279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346a4d3-58ea-42c0-bd7a-50a282208e7f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:51.743882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470f6dd-2ace-4b3b-b00d-a3fc35c08b05 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:51.756308 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d93a1-1c36-45e9-a0cb-1c92130d5607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:51.756985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d93a1-1c36-45e9-a0cb-1c92130d5607 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:51.757319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d93a1-1c36-45e9-a0cb-1c92130d5607 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:51.757604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d93a1-1c36-45e9-a0cb-1c92130d5607 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:51.758403 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d93a1-1c36-45e9-a0cb-1c92130d5607 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:51.758804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d93a1-1c36-45e9-a0cb-1c92130d5607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:51.760220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d93a1-1c36-45e9-a0cb-1c92130d5607 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:51.769440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 468/938] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 4 09:02:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:51.778836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:51.780199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d93a1-1c36-45e9-a0cb-1c92130d5607 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:51.790794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e14d4b-d1e2-4549-b7e1-90f17b2b2481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:51.792130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e14d4b-d1e2-4549-b7e1-90f17b2b2481 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:51.792692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e14d4b-d1e2-4549-b7e1-90f17b2b2481 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:51.793132 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e14d4b-d1e2-4549-b7e1-90f17b2b2481 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:51.804326 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e14d4b-d1e2-4549-b7e1-90f17b2b2481 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:51.806777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e14d4b-d1e2-4549-b7e1-90f17b2b2481 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-90e14d4b-d1e2-4549-b7e1-90f17b2b2481 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:51.824060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:51.825259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:51.826300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:51.826300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:51.826300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:51.826300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:51.826300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:51.826300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:51.826300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:51.826300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:51.833641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 471/939] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:51.835009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e14d4b-d1e2-4549-b7e1-90f17b2b2481 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:51.909050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d93a1-1c36-45e9-a0cb-1c92130d5607 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:51.946213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abff345-bbd6-47ff-ad2a-4ed291e9d83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:51.947309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abff345-bbd6-47ff-ad2a-4ed291e9d83b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:51.947786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abff345-bbd6-47ff-ad2a-4ed291e9d83b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:51.948246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abff345-bbd6-47ff-ad2a-4ed291e9d83b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:51.959849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abff345-bbd6-47ff-ad2a-4ed291e9d83b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:51.961704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abff345-bbd6-47ff-ad2a-4ed291e9d83b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:51.967639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e14d4b-d1e2-4549-b7e1-90f17b2b2481 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-0abff345-bbd6-47ff-ad2a-4ed291e9d83b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:51.979944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:51.980897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:51.981836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:51.981836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:51.981836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:51.981836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:51.984167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 469/940] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:51.989552 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abff345-bbd6-47ff-ad2a-4ed291e9d83b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:52.011617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc8387-982b-4d97-97e8-88f8231fadad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:52.012292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc8387-982b-4d97-97e8-88f8231fadad None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.012551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:52.012668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc8387-982b-4d97-97e8-88f8231fadad None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.012839 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc8387-982b-4d97-97e8-88f8231fadad None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.014071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc8387-982b-4d97-97e8-88f8231fadad None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:52.014853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc8387-982b-4d97-97e8-88f8231fadad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:52.016183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc8387-982b-4d97-97e8-88f8231fadad None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:52.046364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 472/941] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:02:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:52.114300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abff345-bbd6-47ff-ad2a-4ed291e9d83b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:52.144111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c226ea-204c-4975-8873-e5603980c5f7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.144111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c226ea-204c-4975-8873-e5603980c5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.144421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c226ea-204c-4975-8873-e5603980c5f7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.152519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 473/942] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:52.152519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db539d-3840-44c2-b7af-7f7b44b052c9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.152519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db539d-3840-44c2-b7af-7f7b44b052c9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.152519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db539d-3840-44c2-b7af-7f7b44b052c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.169845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545d615-635a-4f62-b2ac-51afdbf4152d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:52.170665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e8db539d-3840-44c2-b7af-7f7b44b052c9 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:52.171217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545d615-635a-4f62-b2ac-51afdbf4152d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.171722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545d615-635a-4f62-b2ac-51afdbf4152d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.172200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e8db539d-3840-44c2-b7af-7f7b44b052c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:52.172397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545d615-635a-4f62-b2ac-51afdbf4152d None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.173344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545d615-635a-4f62-b2ac-51afdbf4152d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:52.173864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545d615-635a-4f62-b2ac-51afdbf4152d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:52.177254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545d615-635a-4f62-b2ac-51afdbf4152d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:52.195082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 470/943] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:52.200870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545d615-635a-4f62-b2ac-51afdbf4152d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:52.226023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:52.252906 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 474/944] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:02:52.276286 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db539d-3840-44c2-b7af-7f7b44b052c9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:52.289572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:52.334078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545d615-635a-4f62-b2ac-51afdbf4152d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:52.370858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b30ca-5748-4f13-aa1b-53f140c9b572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:52.372086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b30ca-5748-4f13-aa1b-53f140c9b572 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.372576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b30ca-5748-4f13-aa1b-53f140c9b572 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.373046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b30ca-5748-4f13-aa1b-53f140c9b572 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.382076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b30ca-5748-4f13-aa1b-53f140c9b572 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:52.383886 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b30ca-5748-4f13-aa1b-53f140c9b572 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-864b30ca-5748-4f13-aa1b-53f140c9b572 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:52.401973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:52.402988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:52.403946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:52.403946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:52.403946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:52.403946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:52.403946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:52.403946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:52.403946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:52.403946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:52.403946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:52.403946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:52.404505 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db539d-3840-44c2-b7af-7f7b44b052c9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:52.406700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 471/945] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 4 09:02:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:52.412955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b30ca-5748-4f13-aa1b-53f140c9b572 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:52.448641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a15eea-d032-438a-a3d2-1b3474d0421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:52.449708 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a15eea-d032-438a-a3d2-1b3474d0421d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.450023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a15eea-d032-438a-a3d2-1b3474d0421d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.450307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a15eea-d032-438a-a3d2-1b3474d0421d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.452035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a15eea-d032-438a-a3d2-1b3474d0421d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:52.453367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a15eea-d032-438a-a3d2-1b3474d0421d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-61a15eea-d032-438a-a3d2-1b3474d0421d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:52.469541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:52.470688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:52.471570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 475/946] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 4 09:02:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:52.476690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a15eea-d032-438a-a3d2-1b3474d0421d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:52.527621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b30ca-5748-4f13-aa1b-53f140c9b572 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:52.536749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:52.559309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:52.566219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffe3d0-de9f-48b5-bc73-bbe9b5c93baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:52.567461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffe3d0-de9f-48b5-bc73-bbe9b5c93baa None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.567969 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffe3d0-de9f-48b5-bc73-bbe9b5c93baa None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.568444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffe3d0-de9f-48b5-bc73-bbe9b5c93baa None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.569498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffe3d0-de9f-48b5-bc73-bbe9b5c93baa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:52.570129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffe3d0-de9f-48b5-bc73-bbe9b5c93baa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:52.571928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffe3d0-de9f-48b5-bc73-bbe9b5c93baa None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:52.591793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 472/947] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 4 09:02:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:52.598014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffe3d0-de9f-48b5-bc73-bbe9b5c93baa None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:52.605918 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a15eea-d032-438a-a3d2-1b3474d0421d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:52.652287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fd957-8c80-44bf-acab-8bef1a94872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:52.652960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fd957-8c80-44bf-acab-8bef1a94872d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.653673 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fd957-8c80-44bf-acab-8bef1a94872d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.653998 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fd957-8c80-44bf-acab-8bef1a94872d None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.654890 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fd957-8c80-44bf-acab-8bef1a94872d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:52.655355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fd957-8c80-44bf-acab-8bef1a94872d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:52.657054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fd957-8c80-44bf-acab-8bef1a94872d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:52.664778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 476/948] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 4 09:02:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:52.671537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fd957-8c80-44bf-acab-8bef1a94872d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:52.709549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffe3d0-de9f-48b5-bc73-bbe9b5c93baa None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:52.745196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562d5f8-dc01-4dde-9224-dae72c007f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:52.746035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562d5f8-dc01-4dde-9224-dae72c007f9a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.746321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562d5f8-dc01-4dde-9224-dae72c007f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.746592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562d5f8-dc01-4dde-9224-dae72c007f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.759936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562d5f8-dc01-4dde-9224-dae72c007f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:52.761517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562d5f8-dc01-4dde-9224-dae72c007f9a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-1562d5f8-dc01-4dde-9224-dae72c007f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:52.780163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:52.781114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:52.782069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:52.782069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:52.782069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:52.782069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:52.786183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 473/949] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:52.787208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562d5f8-dc01-4dde-9224-dae72c007f9a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:52.799788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fd957-8c80-44bf-acab-8bef1a94872d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:52.807578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:52.836203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debff87c-d0eb-41ed-bce7-1adc25856097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:52.836869 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debff87c-d0eb-41ed-bce7-1adc25856097 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.837167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debff87c-d0eb-41ed-bce7-1adc25856097 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.837297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:52.837440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debff87c-d0eb-41ed-bce7-1adc25856097 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.838191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debff87c-d0eb-41ed-bce7-1adc25856097 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:52.838535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debff87c-d0eb-41ed-bce7-1adc25856097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:52.839894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debff87c-d0eb-41ed-bce7-1adc25856097 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:52.844934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 477/950] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 4 09:02:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:52.846191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77028a-b643-441c-88e1-2fb365dff9a0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.846499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77028a-b643-441c-88e1-2fb365dff9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.846867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77028a-b643-441c-88e1-2fb365dff9a0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.848076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 478/951] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:52.850878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77028a-b643-441c-88e1-2fb365dff9a0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:52.899891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562d5f8-dc01-4dde-9224-dae72c007f9a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:52.936664 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b81e2-a433-4f0f-9eb0-171dda2ccb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:52.937338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b81e2-a433-4f0f-9eb0-171dda2ccb56 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.937630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b81e2-a433-4f0f-9eb0-171dda2ccb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.937936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b81e2-a433-4f0f-9eb0-171dda2ccb56 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.938673 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b81e2-a433-4f0f-9eb0-171dda2ccb56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:52.938987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b81e2-a433-4f0f-9eb0-171dda2ccb56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:52.940367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b81e2-a433-4f0f-9eb0-171dda2ccb56 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:52.953925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 474/952] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:52.953925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c58ca-2cdf-4b72-90c2-5c5c19aee52a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:52.953925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c58ca-2cdf-4b72-90c2-5c5c19aee52a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:52.953925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c58ca-2cdf-4b72-90c2-5c5c19aee52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:52.969056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77028a-b643-441c-88e1-2fb365dff9a0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:52.970716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ab0c58ca-2cdf-4b72-90c2-5c5c19aee52a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:52.971906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ab0c58ca-2cdf-4b72-90c2-5c5c19aee52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:53.005874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9db07a-2e5e-4f4f-a7b4-ea04af613a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:53.006556 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9db07a-2e5e-4f4f-a7b4-ea04af613a5b None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:53.006962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9db07a-2e5e-4f4f-a7b4-ea04af613a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:53.007241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9db07a-2e5e-4f4f-a7b4-ea04af613a5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:53.008111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9db07a-2e5e-4f4f-a7b4-ea04af613a5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:53.009484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9db07a-2e5e-4f4f-a7b4-ea04af613a5b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:53.037920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 479/953] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 4 09:02:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:53.037920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9db07a-2e5e-4f4f-a7b4-ea04af613a5b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:53.055911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 475/954] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:53.078815 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c58ca-2cdf-4b72-90c2-5c5c19aee52a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:53.081517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:53.122239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:53.177442 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9db07a-2e5e-4f4f-a7b4-ea04af613a5b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:53.221836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c58ca-2cdf-4b72-90c2-5c5c19aee52a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:53.222045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd9797-3c03-4d68-8e4f-a15f8989f751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:53.223553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd9797-3c03-4d68-8e4f-a15f8989f751 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:53.223843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd9797-3c03-4d68-8e4f-a15f8989f751 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:53.224099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd9797-3c03-4d68-8e4f-a15f8989f751 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:53.231464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd9797-3c03-4d68-8e4f-a15f8989f751 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:53.232924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd9797-3c03-4d68-8e4f-a15f8989f751 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c2bd9797-3c03-4d68-8e4f-a15f8989f751 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:53.251709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:53.253262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:53.255294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:53.255294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:53.255294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:53.255294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:53.255294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:53.255294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:53.255294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:53.255294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:53.255294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 480/955] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:53.260608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd9797-3c03-4d68-8e4f-a15f8989f751 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:53.275673 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd766f-d2e0-415a-8f63-127434ea5e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:53.276932 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd766f-d2e0-415a-8f63-127434ea5e5c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:53.277219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd766f-d2e0-415a-8f63-127434ea5e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:53.277495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd766f-d2e0-415a-8f63-127434ea5e5c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:53.278919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd766f-d2e0-415a-8f63-127434ea5e5c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:53.280507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd766f-d2e0-415a-8f63-127434ea5e5c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-15cd766f-d2e0-415a-8f63-127434ea5e5c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:53.295899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:53.297529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:53.298544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 476/956] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:53.304286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd766f-d2e0-415a-8f63-127434ea5e5c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:53.372253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:53.392091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd9797-3c03-4d68-8e4f-a15f8989f751 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:53.429385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786fb4e-87a6-4bf0-ad4d-559ea57f9cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:53.430070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786fb4e-87a6-4bf0-ad4d-559ea57f9cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:53.430364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786fb4e-87a6-4bf0-ad4d-559ea57f9cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:53.430639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786fb4e-87a6-4bf0-ad4d-559ea57f9cc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:53.431436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786fb4e-87a6-4bf0-ad4d-559ea57f9cc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:53.431747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786fb4e-87a6-4bf0-ad4d-559ea57f9cc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:53.433206 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:53.433321 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786fb4e-87a6-4bf0-ad4d-559ea57f9cc0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:53.447249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd766f-d2e0-415a-8f63-127434ea5e5c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:53.448230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 481/957] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:53.454322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786fb4e-87a6-4bf0-ad4d-559ea57f9cc0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:53.485823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11688e4-a511-4b7e-99ea-ee2d68131047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:53.486441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11688e4-a511-4b7e-99ea-ee2d68131047 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:53.486723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11688e4-a511-4b7e-99ea-ee2d68131047 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:53.486996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11688e4-a511-4b7e-99ea-ee2d68131047 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:53.487768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11688e4-a511-4b7e-99ea-ee2d68131047 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:53.488291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11688e4-a511-4b7e-99ea-ee2d68131047 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:53.489751 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11688e4-a511-4b7e-99ea-ee2d68131047 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:53.499659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 477/958] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:02:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:53.500646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11688e4-a511-4b7e-99ea-ee2d68131047 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:53.569939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786fb4e-87a6-4bf0-ad4d-559ea57f9cc0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:53.614967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30935e-c261-4a6e-8c4b-769835ca6086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:53.616127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30935e-c261-4a6e-8c4b-769835ca6086 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:53.616832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30935e-c261-4a6e-8c4b-769835ca6086 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:53.617119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30935e-c261-4a6e-8c4b-769835ca6086 None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/6bb6d63553794d10a8cba042f8560348/roles/a455148dcf0d4182bf84438cb30422c9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:53.620906 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30935e-c261-4a6e-8c4b-769835ca6086 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=6bb6d63553794d10a8cba042f8560348, role_id=a455148dcf0d4182bf84438cb30422c9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:53.622408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30935e-c261-4a6e-8c4b-769835ca6086 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:53.622767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11688e4-a511-4b7e-99ea-ee2d68131047 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:53.642353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 482/959] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:02:53 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/6bb6d63553794d10a8cba042f8560348/roles/a455148dcf0d4182bf84438cb30422c9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:02:53.667368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293c3dc-c9f6-421a-8b8a-67bcd1bc925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:53.668065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293c3dc-c9f6-421a-8b8a-67bcd1bc925d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:53.668350 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293c3dc-c9f6-421a-8b8a-67bcd1bc925d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:53.668688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293c3dc-c9f6-421a-8b8a-67bcd1bc925d None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:53.669434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293c3dc-c9f6-421a-8b8a-67bcd1bc925d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:53.669744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293c3dc-c9f6-421a-8b8a-67bcd1bc925d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:53.671221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293c3dc-c9f6-421a-8b8a-67bcd1bc925d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:53.677460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 478/960] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:02:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:54.237691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584174-9863-4b4d-84f5-fd481998e550 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:54.238192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584174-9863-4b4d-84f5-fd481998e550 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:54.238587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584174-9863-4b4d-84f5-fd481998e550 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:54.240447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 483/961] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:54.246311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d78fd-cf5c-44c2-8aaa-db6773e6a1f2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:54.246604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d78fd-cf5c-44c2-8aaa-db6773e6a1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:54.246884 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d78fd-cf5c-44c2-8aaa-db6773e6a1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:54.264111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-996d78fd-cf5c-44c2-8aaa-db6773e6a1f2 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:54.265283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-996d78fd-cf5c-44c2-8aaa-db6773e6a1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:54.280168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:54.337997 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 479/962] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:54.359063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584174-9863-4b4d-84f5-fd481998e550 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:54.368928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:54.465682 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584174-9863-4b4d-84f5-fd481998e550 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:54.499323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670143fc-22bc-40c6-b58f-11db329adcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:54.499997 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670143fc-22bc-40c6-b58f-11db329adcd3 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:54.500281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670143fc-22bc-40c6-b58f-11db329adcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:54.500553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670143fc-22bc-40c6-b58f-11db329adcd3 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:54.501268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670143fc-22bc-40c6-b58f-11db329adcd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:54.503066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670143fc-22bc-40c6-b58f-11db329adcd3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:54.517340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.sql.core [None req-670143fc-22bc-40c6-b58f-11db329adcd3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 09:02:54.517340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 09:02:54.517340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [parameters: {'user_id': 'ff880efa3ddf42edb1d025cd150bcf2e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 09:02:54.517340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80284) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-670143fc-22bc-40c6-b58f-11db329adcd3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jul 04 09:02:54.525142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:02:54.526092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application yield resource Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.rollback() Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.force_reraise() Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise self.value Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application session.commit() Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 04 09:02:54.527084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._prepare_impl() Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.flush() Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._flush(objects) Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application compat.raise_( Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application flush_context.execute() Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rec.execute(self) Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 04 09:02:54.528001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 04 09:02:54.528917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 04 09:02:54.529786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ff880efa3ddf42edb1d025cd150bcf2e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:54.532716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 04 09:02:54.534643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 04 09:02:54.536756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 04 09:02:54.536756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 04 09:02:54.536756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 04 09:02:54.536756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 04 09:02:54.536756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 04 09:02:54.536756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:54.536756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 484/963] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 4 09:02:54 2022] POST /identity/v3/users => generated 160 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:54.540770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d78fd-cf5c-44c2-8aaa-db6773e6a1f2 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:54.641577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d78fd-cf5c-44c2-8aaa-db6773e6a1f2 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:54.676271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdf687-d5e9-4ad2-aa43-ffffced2bf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:54.677112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdf687-d5e9-4ad2-aa43-ffffced2bf7c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:54.677411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdf687-d5e9-4ad2-aa43-ffffced2bf7c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:54.677687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdf687-d5e9-4ad2-aa43-ffffced2bf7c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:54.685066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdf687-d5e9-4ad2-aa43-ffffced2bf7c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:54.686549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdf687-d5e9-4ad2-aa43-ffffced2bf7c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-18fdf687-d5e9-4ad2-aa43-ffffced2bf7c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:54.702149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:54.703132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:54.704079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:54.704079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:54.704079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:54.704079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:54.704079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:54.704079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:54.704079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:54.704079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:54.704079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 04 09:02:54.704079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:54.704079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 480/964] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:02:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:54.711235 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670143fc-22bc-40c6-b58f-11db329adcd3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:54.782900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:54.827529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670143fc-22bc-40c6-b58f-11db329adcd3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:54.866072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40adc74-52ce-41db-a3e6-438e2d27f769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:54.866766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40adc74-52ce-41db-a3e6-438e2d27f769 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:54.867053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40adc74-52ce-41db-a3e6-438e2d27f769 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:54.867389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40adc74-52ce-41db-a3e6-438e2d27f769 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:54.868214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40adc74-52ce-41db-a3e6-438e2d27f769 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:54.868524 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40adc74-52ce-41db-a3e6-438e2d27f769 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:54.870204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40adc74-52ce-41db-a3e6-438e2d27f769 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:54.911225 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 485/965] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 4 09:02:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:55.204374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83924132-dd31-4a4b-837c-43736cc07891 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:55.204730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83924132-dd31-4a4b-837c-43736cc07891 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:55.204939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83924132-dd31-4a4b-837c-43736cc07891 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:55.206539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 481/966] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:55.211635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b09d9-eb15-4ed7-a7ec-56e7483358dc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:55.211932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b09d9-eb15-4ed7-a7ec-56e7483358dc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:55.212215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b09d9-eb15-4ed7-a7ec-56e7483358dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:55.229364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4e3b09d9-eb15-4ed7-a7ec-56e7483358dc None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:55.230892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4e3b09d9-eb15-4ed7-a7ec-56e7483358dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:55.298358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 486/967] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:02:55.321019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83924132-dd31-4a4b-837c-43736cc07891 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:55.333730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:55.427684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83924132-dd31-4a4b-837c-43736cc07891 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:55.463606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1bb74-f2b4-41a4-ab52-177604a6e487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:55.464412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1bb74-f2b4-41a4-ab52-177604a6e487 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:55.464697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1bb74-f2b4-41a4-ab52-177604a6e487 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:55.464969 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1bb74-f2b4-41a4-ab52-177604a6e487 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:55.465627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1bb74-f2b4-41a4-ab52-177604a6e487 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:55.467040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1bb74-f2b4-41a4-ab52-177604a6e487 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-52a1bb74-f2b4-41a4-ab52-177604a6e487 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:55.479269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:55.480379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:55.481232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:55.482798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 482/968] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 4 09:02:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:55.486481 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b09d9-eb15-4ed7-a7ec-56e7483358dc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:55.544602 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:55.588935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b09d9-eb15-4ed7-a7ec-56e7483358dc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:55.627979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fc2c9-a534-4717-a95f-bee5e0f661ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:55.628728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fc2c9-a534-4717-a95f-bee5e0f661ea None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:55.629066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fc2c9-a534-4717-a95f-bee5e0f661ea None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:55.629360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fc2c9-a534-4717-a95f-bee5e0f661ea None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:55.630111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fc2c9-a534-4717-a95f-bee5e0f661ea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:55.630428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fc2c9-a534-4717-a95f-bee5e0f661ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:55.631876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fc2c9-a534-4717-a95f-bee5e0f661ea None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:55.637163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 487/969] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 4 09:02:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:55.642888 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1bb74-f2b4-41a4-ab52-177604a6e487 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:55.720000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:55.746532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1bb74-f2b4-41a4-ab52-177604a6e487 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:55.753387 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eabd9-01b2-4647-b0be-453fdb5477c5 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:55.754063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eabd9-01b2-4647-b0be-453fdb5477c5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:55.754063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eabd9-01b2-4647-b0be-453fdb5477c5 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:55.755614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 488/970] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:55.760457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716b0e3-496e-4925-86c4-884e940bae51 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:55.760753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716b0e3-496e-4925-86c4-884e940bae51 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:55.761025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716b0e3-496e-4925-86c4-884e940bae51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:55.777583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b716b0e3-496e-4925-86c4-884e940bae51 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:55.778792 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b716b0e3-496e-4925-86c4-884e940bae51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:55.781656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aeda7-33c7-4955-b8fa-e350bd64a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:55.782357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aeda7-33c7-4955-b8fa-e350bd64a7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:55.782639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aeda7-33c7-4955-b8fa-e350bd64a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:55.782925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aeda7-33c7-4955-b8fa-e350bd64a7d3 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:55.783639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aeda7-33c7-4955-b8fa-e350bd64a7d3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:55.783944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aeda7-33c7-4955-b8fa-e350bd64a7d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:55.785337 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aeda7-33c7-4955-b8fa-e350bd64a7d3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:55.790214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 483/971] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 4 09:02:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:55.790214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc32a52-e863-4d01-9d9d-c777b2721c33 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:55.790214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc32a52-e863-4d01-9d9d-c777b2721c33 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:55.790214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc32a52-e863-4d01-9d9d-c777b2721c33 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:55.790214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 484/972] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:55.797271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc32a52-e863-4d01-9d9d-c777b2721c33 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:55.852251 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 489/973] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:02:55.853938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee0c03-d113-4c81-97fb-445840f75503 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:55.853938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee0c03-d113-4c81-97fb-445840f75503 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:55.854132 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee0c03-d113-4c81-97fb-445840f75503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:55.870406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-70ee0c03-d113-4c81-97fb-445840f75503 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:55.871593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-70ee0c03-d113-4c81-97fb-445840f75503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:55.899881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc32a52-e863-4d01-9d9d-c777b2721c33 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:55.917811 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:55.933270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7140279-268a-4dbe-b996-0415c01c6b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:55.933961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7140279-268a-4dbe-b996-0415c01c6b46 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:55.934246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7140279-268a-4dbe-b996-0415c01c6b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:55.934500 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7140279-268a-4dbe-b996-0415c01c6b46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:55.935304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7140279-268a-4dbe-b996-0415c01c6b46 None admin] RBAC: Adding query filter params (interface=admin, service_id=6f7cd26a348843d09d25a1283276f771, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:55.935618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7140279-268a-4dbe-b996-0415c01c6b46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:55.936992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7140279-268a-4dbe-b996-0415c01c6b46 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:55.939056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 490/974] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:55.939677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee0c03-d113-4c81-97fb-445840f75503 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:55.946694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 485/975] 10.222.0.26 () {56 vars in 1249 bytes} [Mon Jul 4 09:02:55 2022] GET /identity/v3/endpoints?service_id=6f7cd26a348843d09d25a1283276f771&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:55.947736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7140279-268a-4dbe-b996-0415c01c6b46 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:55.950485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:56.042769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee0c03-d113-4c81-97fb-445840f75503 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:56.051425 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7140279-268a-4dbe-b996-0415c01c6b46 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:56.076246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45402ca-5caa-4401-ac61-5a0e82585e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:56.076913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45402ca-5caa-4401-ac61-5a0e82585e72 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:56.077196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45402ca-5caa-4401-ac61-5a0e82585e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:56.077471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45402ca-5caa-4401-ac61-5a0e82585e72 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:56.078230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45402ca-5caa-4401-ac61-5a0e82585e72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:56.079607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45402ca-5caa-4401-ac61-5a0e82585e72 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:56.084490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a40d1-8da7-4a79-a029-e90d47fd612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:56.085167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a40d1-8da7-4a79-a029-e90d47fd612e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:56.085440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a40d1-8da7-4a79-a029-e90d47fd612e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:56.085713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a40d1-8da7-4a79-a029-e90d47fd612e None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:56.086446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a40d1-8da7-4a79-a029-e90d47fd612e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:56.087808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a40d1-8da7-4a79-a029-e90d47fd612e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:56.093209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 491/976] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:02:55 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:02:56.102005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 486/977] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 4 09:02:55 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:02:56.102242 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45402ca-5caa-4401-ac61-5a0e82585e72 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:56.157923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:56.237177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45402ca-5caa-4401-ac61-5a0e82585e72 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:56.276395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dede7e4-c91d-418a-969c-7b8ae9df9b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:56.277223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dede7e4-c91d-418a-969c-7b8ae9df9b13 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:56.277509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dede7e4-c91d-418a-969c-7b8ae9df9b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:56.277785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dede7e4-c91d-418a-969c-7b8ae9df9b13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:56.288408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dede7e4-c91d-418a-969c-7b8ae9df9b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2dede7e4-c91d-418a-969c-7b8ae9df9b13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:02:56.296615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:56.297579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:56.298591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 492/978] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 4 09:02:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:56.303909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a40d1-8da7-4a79-a029-e90d47fd612e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:56.372340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:56.409364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a40d1-8da7-4a79-a029-e90d47fd612e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:56.444121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a2047-3042-4702-9c83-22f9b6c451f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:56.444804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a2047-3042-4702-9c83-22f9b6c451f4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:56.445063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a2047-3042-4702-9c83-22f9b6c451f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:56.445332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a2047-3042-4702-9c83-22f9b6c451f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:56.446122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a2047-3042-4702-9c83-22f9b6c451f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:56.446432 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a2047-3042-4702-9c83-22f9b6c451f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:56.447805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a2047-3042-4702-9c83-22f9b6c451f4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:56.460914 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 487/979] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 4 09:02:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:56.462042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dede7e4-c91d-418a-969c-7b8ae9df9b13 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:56.558156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:56.565511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dede7e4-c91d-418a-969c-7b8ae9df9b13 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:56.600974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785f722-578b-4e88-b5d1-36b83dfe8164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:56.601773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785f722-578b-4e88-b5d1-36b83dfe8164 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:56.602086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785f722-578b-4e88-b5d1-36b83dfe8164 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:56.602356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785f722-578b-4e88-b5d1-36b83dfe8164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:56.613053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785f722-578b-4e88-b5d1-36b83dfe8164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:56.614547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785f722-578b-4e88-b5d1-36b83dfe8164 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a785f722-578b-4e88-b5d1-36b83dfe8164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:56.631755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:56.632688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:56.633591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:56.633591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:56.633591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:56.633591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:56.637782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 493/980] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:56.638696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a2047-3042-4702-9c83-22f9b6c451f4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:56.741379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a2047-3042-4702-9c83-22f9b6c451f4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:56.768375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:56.777445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daaab5-eb7e-41ec-8065-2b84348d1583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:56.778149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daaab5-eb7e-41ec-8065-2b84348d1583 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:56.778431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daaab5-eb7e-41ec-8065-2b84348d1583 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:56.778706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daaab5-eb7e-41ec-8065-2b84348d1583 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:56.779526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86daaab5-eb7e-41ec-8065-2b84348d1583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:56.779835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86daaab5-eb7e-41ec-8065-2b84348d1583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:56.781203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86daaab5-eb7e-41ec-8065-2b84348d1583 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:56.793322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 488/981] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:56.798791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785f722-578b-4e88-b5d1-36b83dfe8164 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:56.900576 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785f722-578b-4e88-b5d1-36b83dfe8164 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:56.933438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2edda-fff9-491d-b495-0a07e3cb4c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:56.934274 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2edda-fff9-491d-b495-0a07e3cb4c76 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:56.934552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2edda-fff9-491d-b495-0a07e3cb4c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:56.934828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2edda-fff9-491d-b495-0a07e3cb4c76 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:56.942469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2edda-fff9-491d-b495-0a07e3cb4c76 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:56.943906 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2edda-fff9-491d-b495-0a07e3cb4c76 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f4f2edda-fff9-491d-b495-0a07e3cb4c76 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:02:56.958381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:56.959341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:56.960337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:56.960337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:56.960337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:02:56.960337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:02:56.960337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:02:56.960337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:02:56.960337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:02:56.960337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:02:56.960337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 04 09:02:56.960337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:56.962203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 494/982] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 4 09:02:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:56.965251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daaab5-eb7e-41ec-8065-2b84348d1583 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:57.070268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daaab5-eb7e-41ec-8065-2b84348d1583 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:57.105549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d87201-416f-411f-b6c9-4ddf05ba2fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:57.106249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87201-416f-411f-b6c9-4ddf05ba2fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:57.106535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87201-416f-411f-b6c9-4ddf05ba2fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:57.106830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87201-416f-411f-b6c9-4ddf05ba2fd8 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:57.107666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d87201-416f-411f-b6c9-4ddf05ba2fd8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:57.107971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d87201-416f-411f-b6c9-4ddf05ba2fd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:57.109389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d87201-416f-411f-b6c9-4ddf05ba2fd8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:57.131761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 489/983] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 4 09:02:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:57.137431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2edda-fff9-491d-b495-0a07e3cb4c76 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:57.153376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:57.257077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2edda-fff9-491d-b495-0a07e3cb4c76 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:57.292512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acdb81-ec98-49b0-a3cf-e91e45a153b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:57.293169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acdb81-ec98-49b0-a3cf-e91e45a153b7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:57.293450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acdb81-ec98-49b0-a3cf-e91e45a153b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:57.293725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acdb81-ec98-49b0-a3cf-e91e45a153b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:57.361329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acdb81-ec98-49b0-a3cf-e91e45a153b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:57.362798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acdb81-ec98-49b0-a3cf-e91e45a153b7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:57.372578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:57.463737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 495/984] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:02:57 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:02:57.469281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d87201-416f-411f-b6c9-4ddf05ba2fd8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:57.517354 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:57.571550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d87201-416f-411f-b6c9-4ddf05ba2fd8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:57.605798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34025c-6feb-4ab0-b276-842e5728513a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:57.606617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34025c-6feb-4ab0-b276-842e5728513a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:57.606900 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34025c-6feb-4ab0-b276-842e5728513a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:57.607176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34025c-6feb-4ab0-b276-842e5728513a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:57.617477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34025c-6feb-4ab0-b276-842e5728513a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:57.618948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34025c-6feb-4ab0-b276-842e5728513a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ab34025c-6feb-4ab0-b276-842e5728513a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:02:57.636706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:57.637612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:57.638555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:02:57.638555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:02:57.638555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:02:57.638555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:57.642218 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 490/985] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:57.643615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acdb81-ec98-49b0-a3cf-e91e45a153b7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:57.746355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acdb81-ec98-49b0-a3cf-e91e45a153b7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:57.756898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:57.783543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31d242-7cea-4638-973c-46f9cd935bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:57.784228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31d242-7cea-4638-973c-46f9cd935bcf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:57.784498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31d242-7cea-4638-973c-46f9cd935bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:57.784800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31d242-7cea-4638-973c-46f9cd935bcf None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:57.785530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31d242-7cea-4638-973c-46f9cd935bcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:57.785834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31d242-7cea-4638-973c-46f9cd935bcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:57.787237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31d242-7cea-4638-973c-46f9cd935bcf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:57.799065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 496/986] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:57.804541 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34025c-6feb-4ab0-b276-842e5728513a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:57.901952 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:57.906873 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34025c-6feb-4ab0-b276-842e5728513a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:57.942455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38886023-f03a-4f34-928b-3f2a86c39699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:57.943246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38886023-f03a-4f34-928b-3f2a86c39699 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:57.943527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38886023-f03a-4f34-928b-3f2a86c39699 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:57.943805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38886023-f03a-4f34-928b-3f2a86c39699 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:57.950623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38886023-f03a-4f34-928b-3f2a86c39699 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:57.952060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38886023-f03a-4f34-928b-3f2a86c39699 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-38886023-f03a-4f34-928b-3f2a86c39699 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:02:57.965715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:57.966639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:57.967537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:02:57.967537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:02:57.967537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:02:57.967537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:02:57.967537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:02:57.967537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:02:57.967537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:02:57.967537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:57.970209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 491/987] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:57.972644 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31d242-7cea-4638-973c-46f9cd935bcf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:58.079247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31d242-7cea-4638-973c-46f9cd935bcf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:58.113706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab4761-ed21-4f6c-802a-daf109a80676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:58.114406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab4761-ed21-4f6c-802a-daf109a80676 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.114690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab4761-ed21-4f6c-802a-daf109a80676 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.114964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab4761-ed21-4f6c-802a-daf109a80676 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.115751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab4761-ed21-4f6c-802a-daf109a80676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:58.116056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab4761-ed21-4f6c-802a-daf109a80676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:58.117485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab4761-ed21-4f6c-802a-daf109a80676 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:58.132611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 497/988] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:02:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:58.138648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab4761-ed21-4f6c-802a-daf109a80676 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:58.139075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d42d8e-9e95-4472-8fc8-49e267328947 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.139297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d42d8e-9e95-4472-8fc8-49e267328947 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.139582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d42d8e-9e95-4472-8fc8-49e267328947 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.141215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 492/989] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:02:58.142275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b4d2e-d953-48c0-8d91-81fa2600bce7 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.142556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b4d2e-d953-48c0-8d91-81fa2600bce7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.142830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b4d2e-d953-48c0-8d91-81fa2600bce7 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.144023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 493/990] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:02:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:02:58.146285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9055d-e0f1-40e3-be01-5972368bedc6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.146570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9055d-e0f1-40e3-be01-5972368bedc6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.146845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9055d-e0f1-40e3-be01-5972368bedc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.163398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f6f9055d-e0f1-40e3-be01-5972368bedc6 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:58.164576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f6f9055d-e0f1-40e3-be01-5972368bedc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:58.167116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:58.233554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 494/991] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:58.234850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6e016-04a7-46f0-8ee0-dd32c68f2f9c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.235128 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6e016-04a7-46f0-8ee0-dd32c68f2f9c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.235408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6e016-04a7-46f0-8ee0-dd32c68f2f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.241508 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab4761-ed21-4f6c-802a-daf109a80676 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:58.255784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-6da6e016-04a7-46f0-8ee0-dd32c68f2f9c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:02:58.257348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6da6e016-04a7-46f0-8ee0-dd32c68f2f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:02:58.276251 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7d5f5-20a5-4e87-9f96-e6d9bfd3028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:58.276905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7d5f5-20a5-4e87-9f96-e6d9bfd3028a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.277586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7d5f5-20a5-4e87-9f96-e6d9bfd3028a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.277867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7d5f5-20a5-4e87-9f96-e6d9bfd3028a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.278711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7d5f5-20a5-4e87-9f96-e6d9bfd3028a None admin] RBAC: Adding query filter params (role.id=a455148dcf0d4182bf84438cb30422c9, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=6bb6d63553794d10a8cba042f8560348) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:58.279022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7d5f5-20a5-4e87-9f96-e6d9bfd3028a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:58.280425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7d5f5-20a5-4e87-9f96-e6d9bfd3028a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:58.295478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 498/992] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:02:58 2022] GET /identity/v3/role_assignments?user.id=6bb6d63553794d10a8cba042f8560348&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=a455148dcf0d4182bf84438cb30422c9 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:58.295478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7d5f5-20a5-4e87-9f96-e6d9bfd3028a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:58.339433 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 495/993] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:02:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:02:58.362618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6e016-04a7-46f0-8ee0-dd32c68f2f9c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:58.382062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:58.388461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:58.400574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7d5f5-20a5-4e87-9f96-e6d9bfd3028a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:58.453335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de60dcd-303a-4a6c-9645-2701dae47f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:58.454191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de60dcd-303a-4a6c-9645-2701dae47f6a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.454535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de60dcd-303a-4a6c-9645-2701dae47f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.454824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de60dcd-303a-4a6c-9645-2701dae47f6a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.455513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de60dcd-303a-4a6c-9645-2701dae47f6a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:58.456938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de60dcd-303a-4a6c-9645-2701dae47f6a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:58.469138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6e016-04a7-46f0-8ee0-dd32c68f2f9c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1de60dcd-303a-4a6c-9645-2701dae47f6a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:58.469423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:58.470450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 04 09:02:58.471312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:02:58.472136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 499/994] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 4 09:02:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:02:58.478238 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de60dcd-303a-4a6c-9645-2701dae47f6a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:58.512063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94cc0a-6375-4e62-9c31-bd27edf5aac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:58.512880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94cc0a-6375-4e62-9c31-bd27edf5aac7 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.513164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94cc0a-6375-4e62-9c31-bd27edf5aac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.513439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94cc0a-6375-4e62-9c31-bd27edf5aac7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.514262 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94cc0a-6375-4e62-9c31-bd27edf5aac7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:58.515638 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94cc0a-6375-4e62-9c31-bd27edf5aac7 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5e94cc0a-6375-4e62-9c31-bd27edf5aac7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:02:58.527875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:02:58.528901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:02:58.529752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 496/995] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:02:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:02:58.535492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94cc0a-6375-4e62-9c31-bd27edf5aac7 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:58.598508 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de60dcd-303a-4a6c-9645-2701dae47f6a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:58.624839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:58.635018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5d0a-f5f9-4f59-a10a-f56cd6fd7c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:58.635682 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5d0a-f5f9-4f59-a10a-f56cd6fd7c6b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.635963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5d0a-f5f9-4f59-a10a-f56cd6fd7c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.636245 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5d0a-f5f9-4f59-a10a-f56cd6fd7c6b None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.636967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5d0a-f5f9-4f59-a10a-f56cd6fd7c6b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:58.637277 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5d0a-f5f9-4f59-a10a-f56cd6fd7c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:58.638701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5d0a-f5f9-4f59-a10a-f56cd6fd7c6b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:58.643667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 500/996] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 4 09:02:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:58.646812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94cc0a-6375-4e62-9c31-bd27edf5aac7 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:58.649280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5d0a-f5f9-4f59-a10a-f56cd6fd7c6b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:58.673047 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:58.684809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfb3be-954c-47e0-9f96-c3636c7fabfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:58.685731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb3be-954c-47e0-9f96-c3636c7fabfb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.686046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb3be-954c-47e0-9f96-c3636c7fabfb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.686324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb3be-954c-47e0-9f96-c3636c7fabfb None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.687049 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfb3be-954c-47e0-9f96-c3636c7fabfb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:58.687501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfb3be-954c-47e0-9f96-c3636c7fabfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:58.689662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfb3be-954c-47e0-9f96-c3636c7fabfb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:58.699639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 497/997] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:02:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:58.701037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfb3be-954c-47e0-9f96-c3636c7fabfb None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:58.756708 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5d0a-f5f9-4f59-a10a-f56cd6fd7c6b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:58.794693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c95c7-04ad-4086-937c-824d291705af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:58.795377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c95c7-04ad-4086-937c-824d291705af None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.795948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c95c7-04ad-4086-937c-824d291705af None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.796369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c95c7-04ad-4086-937c-824d291705af None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.796945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c95c7-04ad-4086-937c-824d291705af None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:58.797253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c95c7-04ad-4086-937c-824d291705af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:58.798647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c95c7-04ad-4086-937c-824d291705af None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:58.804231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 501/998] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 4 09:02:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:02:58.809577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c95c7-04ad-4086-937c-824d291705af None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:58.817697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfb3be-954c-47e0-9f96-c3636c7fabfb None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:58.853162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa5642-fb07-47a4-a51c-edb60962b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:58.854426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5642-fb07-47a4-a51c-edb60962b91a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.854712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5642-fb07-47a4-a51c-edb60962b91a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.855375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5642-fb07-47a4-a51c-edb60962b91a None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.856110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa5642-fb07-47a4-a51c-edb60962b91a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:58.856421 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa5642-fb07-47a4-a51c-edb60962b91a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:58.857792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa5642-fb07-47a4-a51c-edb60962b91a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:58.863054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 498/999] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:02:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:58.868779 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa5642-fb07-47a4-a51c-edb60962b91a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:58.882998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:58.900529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:02:58.921203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c95c7-04ad-4086-937c-824d291705af None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:58.955514 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39507f2-b7b2-434f-8d1b-0cd4d57b0313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:58.956190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39507f2-b7b2-434f-8d1b-0cd4d57b0313 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:58.956469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39507f2-b7b2-434f-8d1b-0cd4d57b0313 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:58.956831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39507f2-b7b2-434f-8d1b-0cd4d57b0313 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:58.957583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39507f2-b7b2-434f-8d1b-0cd4d57b0313 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:58.958988 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39507f2-b7b2-434f-8d1b-0cd4d57b0313 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:58.975319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 502/1000] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 4 09:02:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:02:58.975529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa5642-fb07-47a4-a51c-edb60962b91a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:59.020842 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b5a4f-01c7-4fc2-bfc9-7c01311b047e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:59.021519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b5a4f-01c7-4fc2-bfc9-7c01311b047e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:59.021809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b5a4f-01c7-4fc2-bfc9-7c01311b047e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:59.022114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b5a4f-01c7-4fc2-bfc9-7c01311b047e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:59.022929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b5a4f-01c7-4fc2-bfc9-7c01311b047e None admin] RBAC: Adding query filter params (interface=public, service_id=3d460ab65fa84fe493dd1803e2892b47, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:02:59.023256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b5a4f-01c7-4fc2-bfc9-7c01311b047e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:59.024662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b5a4f-01c7-4fc2-bfc9-7c01311b047e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:59.036188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 499/1001] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 4 09:02:58 2022] GET /identity/v3/endpoints?service_id=3d460ab65fa84fe493dd1803e2892b47&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:02:59.037232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39507f2-b7b2-434f-8d1b-0cd4d57b0313 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:02:59.153518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39507f2-b7b2-434f-8d1b-0cd4d57b0313 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:02:59.188214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2fa22-1545-4a30-9904-d20c77d806ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:02:59.188891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2fa22-1545-4a30-9904-d20c77d806ac None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:02:59.189180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2fa22-1545-4a30-9904-d20c77d806ac None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:02:59.189454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2fa22-1545-4a30-9904-d20c77d806ac None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:02:59.190170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2fa22-1545-4a30-9904-d20c77d806ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:02:59.191624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2fa22-1545-4a30-9904-d20c77d806ac None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:02:59.198092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 503/1002] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 4 09:02:59 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:03:00.381441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3641ed-b699-4c9a-928c-8b121ff86b95 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:00.382062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3641ed-b699-4c9a-928c-8b121ff86b95 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:00.382062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3641ed-b699-4c9a-928c-8b121ff86b95 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:00.383536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 500/1003] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:03:00.388610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570fb30-9ecb-485e-a7b8-d9a6fa48bb40 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:00.389051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570fb30-9ecb-485e-a7b8-d9a6fa48bb40 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:00.389143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570fb30-9ecb-485e-a7b8-d9a6fa48bb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:00.406417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2570fb30-9ecb-485e-a7b8-d9a6fa48bb40 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:00.407518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2570fb30-9ecb-485e-a7b8-d9a6fa48bb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:00.409589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:00.478411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 504/1004] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:00 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:00.500447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3641ed-b699-4c9a-928c-8b121ff86b95 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:00.583451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:00.625526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3641ed-b699-4c9a-928c-8b121ff86b95 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:00.663564 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222c137-dcd8-4ad8-83fa-fa8c3f0336eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:00.664324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222c137-dcd8-4ad8-83fa-fa8c3f0336eb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:00.664605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222c137-dcd8-4ad8-83fa-fa8c3f0336eb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:00.664873 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222c137-dcd8-4ad8-83fa-fa8c3f0336eb None admin] PATH_INFO: `/v3/services/image` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:00.665525 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222c137-dcd8-4ad8-83fa-fa8c3f0336eb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:00.666909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222c137-dcd8-4ad8-83fa-fa8c3f0336eb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-e222c137-dcd8-4ad8-83fa-fa8c3f0336eb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:00.678892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:00.679915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:03:00.680776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:00.684948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 501/1005] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jul 4 09:03:00 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:00.686261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570fb30-9ecb-485e-a7b8-d9a6fa48bb40 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:00.791635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570fb30-9ecb-485e-a7b8-d9a6fa48bb40 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:00.826489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56ecfc-9eaa-4d37-bc8b-b8f7f9546306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:00.827159 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56ecfc-9eaa-4d37-bc8b-b8f7f9546306 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:00.827444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56ecfc-9eaa-4d37-bc8b-b8f7f9546306 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:00.827719 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56ecfc-9eaa-4d37-bc8b-b8f7f9546306 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:00.828438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56ecfc-9eaa-4d37-bc8b-b8f7f9546306 None admin] RBAC: Adding query filter params (name=image) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:00.828747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56ecfc-9eaa-4d37-bc8b-b8f7f9546306 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:00.830148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56ecfc-9eaa-4d37-bc8b-b8f7f9546306 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:00.835416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 505/1006] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:00.840909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222c137-dcd8-4ad8-83fa-fa8c3f0336eb None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:00.961302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222c137-dcd8-4ad8-83fa-fa8c3f0336eb None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:00.997162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666391f2-a07c-4a8c-9104-30693110c5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:00.997920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666391f2-a07c-4a8c-9104-30693110c5bb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:00.998173 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666391f2-a07c-4a8c-9104-30693110c5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:00.998474 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666391f2-a07c-4a8c-9104-30693110c5bb None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:00.999216 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666391f2-a07c-4a8c-9104-30693110c5bb None admin] RBAC: Adding query filter params (type=image) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:00.999522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666391f2-a07c-4a8c-9104-30693110c5bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:01.000881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666391f2-a07c-4a8c-9104-30693110c5bb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:01.002176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:01.014213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 502/1007] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:01.015794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c3511-b070-4f4f-b3f5-a7231e94da5d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:01.016083 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c3511-b070-4f4f-b3f5-a7231e94da5d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:01.016359 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c3511-b070-4f4f-b3f5-a7231e94da5d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:01.017834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 506/1008] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:01.023880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa827-8de5-4134-ba1f-972807897b03 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:01.024175 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa827-8de5-4134-ba1f-972807897b03 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:01.024455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa827-8de5-4134-ba1f-972807897b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:01.042854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-691aa827-8de5-4134-ba1f-972807897b03 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:01.044059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-691aa827-8de5-4134-ba1f-972807897b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:01.118809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 503/1009] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:01.142531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c3511-b070-4f4f-b3f5-a7231e94da5d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:01.145874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:01.199229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6aebeb-616d-4d62-8165-b61a09175d2f None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:01.199793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6aebeb-616d-4d62-8165-b61a09175d2f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:01.200090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6aebeb-616d-4d62-8165-b61a09175d2f None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:01.201934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 504/1010] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:01.206865 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339df24-5cae-4437-9a02-d20e082cb1c5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:01.207263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339df24-5cae-4437-9a02-d20e082cb1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:01.207549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339df24-5cae-4437-9a02-d20e082cb1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:01.224506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b339df24-5cae-4437-9a02-d20e082cb1c5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:01.225611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b339df24-5cae-4437-9a02-d20e082cb1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:01.259361 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c3511-b070-4f4f-b3f5-a7231e94da5d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:01.296242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 505/1011] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:01.297345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ed05a-44ea-496c-bbff-945bd160494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:01.298338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ed05a-44ea-496c-bbff-945bd160494d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:01.298759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ed05a-44ea-496c-bbff-945bd160494d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:01.299190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ed05a-44ea-496c-bbff-945bd160494d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:01.300539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ed05a-44ea-496c-bbff-945bd160494d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:01.302156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ed05a-44ea-496c-bbff-945bd160494d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:01.304246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 507/1012] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:01.309830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339df24-5cae-4437-9a02-d20e082cb1c5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:01.319370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ed05a-44ea-496c-bbff-945bd160494d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:01.371876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:01.390946 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:01.418879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339df24-5cae-4437-9a02-d20e082cb1c5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:01.422025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ed05a-44ea-496c-bbff-945bd160494d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:01.454622 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f96e9-e627-4bca-8749-eb3c084ba313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:01.455701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f96e9-e627-4bca-8749-eb3c084ba313 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:01.456255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f96e9-e627-4bca-8749-eb3c084ba313 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:01.456732 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f96e9-e627-4bca-8749-eb3c084ba313 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:01.457218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263abb45-1757-444b-b14d-0fd8b26e077e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:01.457847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f96e9-e627-4bca-8749-eb3c084ba313 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:01.458065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263abb45-1757-444b-b14d-0fd8b26e077e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:01.458345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263abb45-1757-444b-b14d-0fd8b26e077e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:01.458611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263abb45-1757-444b-b14d-0fd8b26e077e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:01.459262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263abb45-1757-444b-b14d-0fd8b26e077e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:01.460227 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f96e9-e627-4bca-8749-eb3c084ba313 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:01.460607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263abb45-1757-444b-b14d-0fd8b26e077e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-263abb45-1757-444b-b14d-0fd8b26e077e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:01.472820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:01.474170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:01.475348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 508/1013] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:03:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:03:01.479504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263abb45-1757-444b-b14d-0fd8b26e077e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:01.493923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 506/1014] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:03:01 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:03:01.591952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263abb45-1757-444b-b14d-0fd8b26e077e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:01.628072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b22a0-37b0-4293-9480-012b5c26efa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:01.628961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b22a0-37b0-4293-9480-012b5c26efa9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:01.629273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b22a0-37b0-4293-9480-012b5c26efa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:01.629615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b22a0-37b0-4293-9480-012b5c26efa9 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:01.630320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b22a0-37b0-4293-9480-012b5c26efa9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:01.630620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b22a0-37b0-4293-9480-012b5c26efa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:01.632118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b22a0-37b0-4293-9480-012b5c26efa9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:01.633454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:01.643076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 509/1015] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:03:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:01.645117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f96e9-e627-4bca-8749-eb3c084ba313 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:01.750988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f96e9-e627-4bca-8749-eb3c084ba313 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:01.762054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:01.789538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43acf64-e7e9-40aa-bfba-b343bdce102a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:01.790240 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43acf64-e7e9-40aa-bfba-b343bdce102a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:01.790523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43acf64-e7e9-40aa-bfba-b343bdce102a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:01.790794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43acf64-e7e9-40aa-bfba-b343bdce102a None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:01.791503 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43acf64-e7e9-40aa-bfba-b343bdce102a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:01.791811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43acf64-e7e9-40aa-bfba-b343bdce102a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:01.793181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43acf64-e7e9-40aa-bfba-b343bdce102a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:01.798407 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 507/1016] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 4 09:03:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:01.804419 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b22a0-37b0-4293-9480-012b5c26efa9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:01.908604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b22a0-37b0-4293-9480-012b5c26efa9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:01.942077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e2ead-2aa5-4b73-8df2-e0c7d38abd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:01.942746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e2ead-2aa5-4b73-8df2-e0c7d38abd90 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:01.943028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e2ead-2aa5-4b73-8df2-e0c7d38abd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:01.943302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e2ead-2aa5-4b73-8df2-e0c7d38abd90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:01.944062 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e2ead-2aa5-4b73-8df2-e0c7d38abd90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:01.945424 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e2ead-2aa5-4b73-8df2-e0c7d38abd90 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:01.970204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 510/1017] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 4 09:03:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:03:03.057323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d96bb-bd72-422c-bea7-9751e049c744 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:03.058123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d96bb-bd72-422c-bea7-9751e049c744 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:03.058123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d96bb-bd72-422c-bea7-9751e049c744 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:03.059411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 508/1018] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:03.064454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2d0ed-029e-4b85-ae76-745d23333f59 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:03.064903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2d0ed-029e-4b85-ae76-745d23333f59 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:03.065047 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2d0ed-029e-4b85-ae76-745d23333f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:03.082846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2dd2d0ed-029e-4b85-ae76-745d23333f59 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:03.084040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2dd2d0ed-029e-4b85-ae76-745d23333f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:03.139619 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:03.170675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 511/1019] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:03 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:03.197717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d96bb-bd72-422c-bea7-9751e049c744 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:03.300603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d96bb-bd72-422c-bea7-9751e049c744 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:03.335564 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78eaf5-8204-444e-8942-870d795d8a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:03.336234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78eaf5-8204-444e-8942-870d795d8a11 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:03.336510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78eaf5-8204-444e-8942-870d795d8a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:03.336781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78eaf5-8204-444e-8942-870d795d8a11 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:03.337519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78eaf5-8204-444e-8942-870d795d8a11 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:03.338920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78eaf5-8204-444e-8942-870d795d8a11 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:03.353795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 509/1020] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:03:03 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:03:03.562274 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66da53-e741-4c8c-bc03-008d47136580 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:03.562618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66da53-e741-4c8c-bc03-008d47136580 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:03.562858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66da53-e741-4c8c-bc03-008d47136580 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:03.564371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 512/1021] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:03.569991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30ad7c-a841-4c87-bf98-52903afb8b00 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:03.570235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30ad7c-a841-4c87-bf98-52903afb8b00 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:03.570516 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30ad7c-a841-4c87-bf98-52903afb8b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:03.587721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ef30ad7c-a841-4c87-bf98-52903afb8b00 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:03.588810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ef30ad7c-a841-4c87-bf98-52903afb8b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:03.616805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:03.666645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 510/1022] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:03:03.691447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66da53-e741-4c8c-bc03-008d47136580 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:03.800923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66da53-e741-4c8c-bc03-008d47136580 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:03.835371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93b89b-b7cb-4f94-9fe6-47cfd94cd73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:03.836171 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93b89b-b7cb-4f94-9fe6-47cfd94cd73f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:03.836454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93b89b-b7cb-4f94-9fe6-47cfd94cd73f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:03.836733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93b89b-b7cb-4f94-9fe6-47cfd94cd73f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:03.837995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93b89b-b7cb-4f94-9fe6-47cfd94cd73f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:03.839387 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93b89b-b7cb-4f94-9fe6-47cfd94cd73f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:03.841205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 513/1023] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:03.846803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30ad7c-a841-4c87-bf98-52903afb8b00 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:03.951632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30ad7c-a841-4c87-bf98-52903afb8b00 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:03.995218 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba52bd-0ee0-4be8-8d06-98c434d07900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:03.995889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba52bd-0ee0-4be8-8d06-98c434d07900 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:03.996168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba52bd-0ee0-4be8-8d06-98c434d07900 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:03.996436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba52bd-0ee0-4be8-8d06-98c434d07900 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:03.997148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba52bd-0ee0-4be8-8d06-98c434d07900 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:03.998621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba52bd-0ee0-4be8-8d06-98c434d07900 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:04.029911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 511/1024] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:03:03 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:03:04.112544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dcbc8-efc1-45d3-89ee-7449c0e38764 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:04.113142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dcbc8-efc1-45d3-89ee-7449c0e38764 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:04.113301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dcbc8-efc1-45d3-89ee-7449c0e38764 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:04.115173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 514/1025] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:04.120593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d0cd1-2d7b-4de4-bc75-5856b5073cd1 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:04.121081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d0cd1-2d7b-4de4-bc75-5856b5073cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:04.121156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d0cd1-2d7b-4de4-bc75-5856b5073cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:04.140224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-dd3d0cd1-2d7b-4de4-bc75-5856b5073cd1 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:04.141460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-dd3d0cd1-2d7b-4de4-bc75-5856b5073cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:04.157287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:04.219703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 512/1026] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:04.243319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dcbc8-efc1-45d3-89ee-7449c0e38764 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:04.298496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:04.355115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dcbc8-efc1-45d3-89ee-7449c0e38764 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:04.391324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af84d7-7a4c-412c-8dac-cae8df47318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:04.392071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af84d7-7a4c-412c-8dac-cae8df47318b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:04.392405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af84d7-7a4c-412c-8dac-cae8df47318b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:04.392687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af84d7-7a4c-412c-8dac-cae8df47318b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:04.403734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af84d7-7a4c-412c-8dac-cae8df47318b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c5af84d7-7a4c-412c-8dac-cae8df47318b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:04.412823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:04.413787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:04.414684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 515/1027] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 4 09:03:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:04.421556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d0cd1-2d7b-4de4-bc75-5856b5073cd1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:04.533754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d0cd1-2d7b-4de4-bc75-5856b5073cd1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:04.576844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32922c87-ffc7-4ec4-ae1a-42e06e353860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:04.577526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32922c87-ffc7-4ec4-ae1a-42e06e353860 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:04.577814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32922c87-ffc7-4ec4-ae1a-42e06e353860 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:04.578138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32922c87-ffc7-4ec4-ae1a-42e06e353860 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:04.578894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32922c87-ffc7-4ec4-ae1a-42e06e353860 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:04.579701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32922c87-ffc7-4ec4-ae1a-42e06e353860 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:04.581093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32922c87-ffc7-4ec4-ae1a-42e06e353860 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:04.590474 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 513/1028] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 4 09:03:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:04.596205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af84d7-7a4c-412c-8dac-cae8df47318b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:04.710395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af84d7-7a4c-412c-8dac-cae8df47318b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:04.719743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:04.745991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b494e1-2d79-4db0-9ffe-83ad91080220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:04.746783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b494e1-2d79-4db0-9ffe-83ad91080220 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:04.747061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b494e1-2d79-4db0-9ffe-83ad91080220 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:04.747333 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b494e1-2d79-4db0-9ffe-83ad91080220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:04.758382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b494e1-2d79-4db0-9ffe-83ad91080220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:04.759819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b494e1-2d79-4db0-9ffe-83ad91080220 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c8b494e1-2d79-4db0-9ffe-83ad91080220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:03:04.776872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:04.777842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:04.778791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:03:04.778791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:03:04.778791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:04.778791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:04.782226 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 516/1029] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:04.784229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32922c87-ffc7-4ec4-ae1a-42e06e353860 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:04.796349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:04.893696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32922c87-ffc7-4ec4-ae1a-42e06e353860 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:04.927426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ade8e7-b4c5-4b37-839d-e61f6eaa72d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:04.928110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ade8e7-b4c5-4b37-839d-e61f6eaa72d3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:04.928392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ade8e7-b4c5-4b37-839d-e61f6eaa72d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:04.928666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ade8e7-b4c5-4b37-839d-e61f6eaa72d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:04.929394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ade8e7-b4c5-4b37-839d-e61f6eaa72d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:04.929702 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ade8e7-b4c5-4b37-839d-e61f6eaa72d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:04.931110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ade8e7-b4c5-4b37-839d-e61f6eaa72d3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:04.943435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 514/1030] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:04.948841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b494e1-2d79-4db0-9ffe-83ad91080220 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:05.053647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b494e1-2d79-4db0-9ffe-83ad91080220 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:05.089844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6d664-e228-4ac2-9a7e-5c19e77dfe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:05.090675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6d664-e228-4ac2-9a7e-5c19e77dfe19 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:05.090958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6d664-e228-4ac2-9a7e-5c19e77dfe19 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:05.091240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6d664-e228-4ac2-9a7e-5c19e77dfe19 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:05.099342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6d664-e228-4ac2-9a7e-5c19e77dfe19 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:05.100792 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6d664-e228-4ac2-9a7e-5c19e77dfe19 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8fc6d664-e228-4ac2-9a7e-5c19e77dfe19 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:03:05.114680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:05.116079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:05.117482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:05.117482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:05.117482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:03:05.117482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:03:05.117482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:03:05.117482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:03:05.117482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:03:05.117482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:03:05.117482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:03:05.117482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:05.121971 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 517/1031] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 4 09:03:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:05.122107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ade8e7-b4c5-4b37-839d-e61f6eaa72d3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:05.196277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:05.234122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ade8e7-b4c5-4b37-839d-e61f6eaa72d3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:05.269640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6a21f-2924-4ccd-9689-f96ce33f5bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:05.270419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6a21f-2924-4ccd-9689-f96ce33f5bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:05.270701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6a21f-2924-4ccd-9689-f96ce33f5bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:05.270977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6a21f-2924-4ccd-9689-f96ce33f5bb6 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:05.271825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6a21f-2924-4ccd-9689-f96ce33f5bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:05.272135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6a21f-2924-4ccd-9689-f96ce33f5bb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:05.273568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6a21f-2924-4ccd-9689-f96ce33f5bb6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:05.291176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 515/1032] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 4 09:03:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:05.296822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6d664-e228-4ac2-9a7e-5c19e77dfe19 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:05.319736 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:05.373134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a1b21-7ed2-4469-bdd4-e423fd841261 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:05.373491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a1b21-7ed2-4469-bdd4-e423fd841261 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:05.373681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a1b21-7ed2-4469-bdd4-e423fd841261 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:05.375220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 516/1033] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:03:05.380564 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95e1e8-91cb-4a7b-9c15-99974a5a2dfa None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:05.380844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95e1e8-91cb-4a7b-9c15-99974a5a2dfa None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:05.381120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95e1e8-91cb-4a7b-9c15-99974a5a2dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:05.397769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7b95e1e8-91cb-4a7b-9c15-99974a5a2dfa None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:05.398986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7b95e1e8-91cb-4a7b-9c15-99974a5a2dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:05.403797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6d664-e228-4ac2-9a7e-5c19e77dfe19 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:05.439957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681fb58-e4fb-460f-9696-55c63011f01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:05.440605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681fb58-e4fb-460f-9696-55c63011f01a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:05.440887 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681fb58-e4fb-460f-9696-55c63011f01a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:05.441207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681fb58-e4fb-460f-9696-55c63011f01a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:05.473919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 517/1034] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:03:05.494452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95e1e8-91cb-4a7b-9c15-99974a5a2dfa None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:05.511626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681fb58-e4fb-460f-9696-55c63011f01a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:05.513094 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681fb58-e4fb-460f-9696-55c63011f01a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:05.533849 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:05.554165 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:05.602507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95e1e8-91cb-4a7b-9c15-99974a5a2dfa None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:05.617633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 518/1035] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:05.623889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681fb58-e4fb-460f-9696-55c63011f01a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:05.643592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cd196-a823-49e7-a116-3db986f72f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:05.644413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cd196-a823-49e7-a116-3db986f72f4d None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:05.644694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cd196-a823-49e7-a116-3db986f72f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:05.644966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cd196-a823-49e7-a116-3db986f72f4d None admin] PATH_INFO: `/v3/services/image` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:05.645629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891cd196-a823-49e7-a116-3db986f72f4d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:05.647029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891cd196-a823-49e7-a116-3db986f72f4d None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-891cd196-a823-49e7-a116-3db986f72f4d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:05.658568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:05.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:05.661008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 518/1036] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jul 4 09:03:05 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:03:05.666830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cd196-a823-49e7-a116-3db986f72f4d None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:05.733123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681fb58-e4fb-460f-9696-55c63011f01a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:05.755420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:05.770340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90133f8-f912-47aa-9e8a-4e2242ad7861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:05.771143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90133f8-f912-47aa-9e8a-4e2242ad7861 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:05.771426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90133f8-f912-47aa-9e8a-4e2242ad7861 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:05.771703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90133f8-f912-47aa-9e8a-4e2242ad7861 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:05.771969 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cd196-a823-49e7-a116-3db986f72f4d None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:05.783163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90133f8-f912-47aa-9e8a-4e2242ad7861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:05.784609 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90133f8-f912-47aa-9e8a-4e2242ad7861 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:05.795370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c90133f8-f912-47aa-9e8a-4e2242ad7861 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:03:05.803478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:05.804448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:05.805339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:03:05.805339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:03:05.805339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:05.805339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:05.805339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 519/1037] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:03:05.810018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90133f8-f912-47aa-9e8a-4e2242ad7861 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:05.814917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c742292-456d-47ca-88a2-33da388ecb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:05.815596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742292-456d-47ca-88a2-33da388ecb36 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:05.815882 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742292-456d-47ca-88a2-33da388ecb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:05.816224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742292-456d-47ca-88a2-33da388ecb36 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:05.816981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c742292-456d-47ca-88a2-33da388ecb36 None admin] RBAC: Adding query filter params (name=image) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:05.817333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c742292-456d-47ca-88a2-33da388ecb36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:05.819147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c742292-456d-47ca-88a2-33da388ecb36 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:05.829733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 519/1038] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:05.830648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c742292-456d-47ca-88a2-33da388ecb36 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:05.912790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90133f8-f912-47aa-9e8a-4e2242ad7861 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:05.937775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c742292-456d-47ca-88a2-33da388ecb36 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:05.949161 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374db2fd-f017-432a-bd23-f4e01b62b187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:05.949829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374db2fd-f017-432a-bd23-f4e01b62b187 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:05.950140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374db2fd-f017-432a-bd23-f4e01b62b187 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:05.950416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374db2fd-f017-432a-bd23-f4e01b62b187 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:05.951145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374db2fd-f017-432a-bd23-f4e01b62b187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:05.951454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374db2fd-f017-432a-bd23-f4e01b62b187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:05.952822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374db2fd-f017-432a-bd23-f4e01b62b187 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:05.965074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 520/1039] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:05.970507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374db2fd-f017-432a-bd23-f4e01b62b187 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:05.977958 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb8855-2290-4bc8-a959-0a688db43a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:05.978625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb8855-2290-4bc8-a959-0a688db43a81 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:05.978907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb8855-2290-4bc8-a959-0a688db43a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:05.979174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb8855-2290-4bc8-a959-0a688db43a81 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:05.979880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb8855-2290-4bc8-a959-0a688db43a81 None admin] RBAC: Adding query filter params (type=image) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:05.980181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb8855-2290-4bc8-a959-0a688db43a81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:05.981670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb8855-2290-4bc8-a959-0a688db43a81 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:05.986732 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 520/1040] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:05.992536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb8855-2290-4bc8-a959-0a688db43a81 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:06.003462 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:06.043317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:06.080169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374db2fd-f017-432a-bd23-f4e01b62b187 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:06.095997 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb8855-2290-4bc8-a959-0a688db43a81 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:06.116680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ebe33-b30f-4fe7-af44-4532daae36c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:06.117481 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ebe33-b30f-4fe7-af44-4532daae36c2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:06.117766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ebe33-b30f-4fe7-af44-4532daae36c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:06.118075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ebe33-b30f-4fe7-af44-4532daae36c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:06.125362 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ebe33-b30f-4fe7-af44-4532daae36c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:06.127057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ebe33-b30f-4fe7-af44-4532daae36c2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:06.129794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45282991-4a17-46f1-bf2d-0db48cb2f617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:06.130484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45282991-4a17-46f1-bf2d-0db48cb2f617 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:06.130772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45282991-4a17-46f1-bf2d-0db48cb2f617 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:06.131028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45282991-4a17-46f1-bf2d-0db48cb2f617 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:06.131968 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45282991-4a17-46f1-bf2d-0db48cb2f617 None admin] RBAC: Adding query filter params (interface=public, service_id=7de298a4b2734294b4c462c6423d186a, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:06.132281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45282991-4a17-46f1-bf2d-0db48cb2f617 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:06.133628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45282991-4a17-46f1-bf2d-0db48cb2f617 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:06.138797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 521/1041] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 4 09:03:05 2022] GET /identity/v3/endpoints?service_id=7de298a4b2734294b4c462c6423d186a&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:06.140064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab20c0d-2af5-46aa-8637-627515aea020 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:06.140348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab20c0d-2af5-46aa-8637-627515aea020 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:06.140621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab20c0d-2af5-46aa-8637-627515aea020 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:06.142006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 522/1042] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d48ebe33-b30f-4fe7-af44-4532daae36c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:03:06.142205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:06.143114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:06.144093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:03:06.144093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:03:06.144093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:03:06.144093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:03:06.144093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:03:06.144093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:03:06.144093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:03:06.144093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:06.144093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 521/1043] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:03:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:03:06.146798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab20c0d-2af5-46aa-8637-627515aea020 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:06.148382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f793f7-a9e4-43cf-837a-3aaee74de18e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:06.148666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f793f7-a9e4-43cf-837a-3aaee74de18e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:06.148943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f793f7-a9e4-43cf-837a-3aaee74de18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:06.164846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-66f793f7-a9e4-43cf-837a-3aaee74de18e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:06.166007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-66f793f7-a9e4-43cf-837a-3aaee74de18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:06.230512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 522/1044] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:06.231343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f793f7-a9e4-43cf-837a-3aaee74de18e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:06.255738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:06.257831 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab20c0d-2af5-46aa-8637-627515aea020 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:06.271642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:06.292778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991bb3f-f16d-4418-8b09-f3df020103ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:06.293413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991bb3f-f16d-4418-8b09-f3df020103ea None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:06.293690 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991bb3f-f16d-4418-8b09-f3df020103ea None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:06.293995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991bb3f-f16d-4418-8b09-f3df020103ea None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:06.294674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991bb3f-f16d-4418-8b09-f3df020103ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:06.296033 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991bb3f-f16d-4418-8b09-f3df020103ea None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:06.302561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 523/1045] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 4 09:03:06 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:03:06.304426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991bb3f-f16d-4418-8b09-f3df020103ea None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:06.334616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f793f7-a9e4-43cf-837a-3aaee74de18e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:06.378931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9d08a-e547-42fa-95e0-2f7eace99902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:06.379918 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9d08a-e547-42fa-95e0-2f7eace99902 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:06.380231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9d08a-e547-42fa-95e0-2f7eace99902 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:06.380491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9d08a-e547-42fa-95e0-2f7eace99902 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:06.381320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9d08a-e547-42fa-95e0-2f7eace99902 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:06.381633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9d08a-e547-42fa-95e0-2f7eace99902 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:06.383143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9d08a-e547-42fa-95e0-2f7eace99902 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:06.397697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 523/1046] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:03:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:06.403395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9d08a-e547-42fa-95e0-2f7eace99902 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:06.430881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991bb3f-f16d-4418-8b09-f3df020103ea None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:06.471483 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70aa0a5-b667-4d56-9d27-9bee732ad824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:06.472372 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70aa0a5-b667-4d56-9d27-9bee732ad824 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:06.472658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70aa0a5-b667-4d56-9d27-9bee732ad824 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:06.472946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70aa0a5-b667-4d56-9d27-9bee732ad824 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:06.485543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70aa0a5-b667-4d56-9d27-9bee732ad824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c70aa0a5-b667-4d56-9d27-9bee732ad824 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:06.493466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:06.494670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:06.495658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 524/1047] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:03:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:06.498638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:06.500914 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70aa0a5-b667-4d56-9d27-9bee732ad824 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:06.517036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9d08a-e547-42fa-95e0-2f7eace99902 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:06.539169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:06.559114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ce835-5a43-45e0-be83-47a202abb828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:06.559775 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ce835-5a43-45e0-be83-47a202abb828 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:06.560058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ce835-5a43-45e0-be83-47a202abb828 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:06.560332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ce835-5a43-45e0-be83-47a202abb828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:06.561167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ce835-5a43-45e0-be83-47a202abb828 None admin] RBAC: Adding query filter params (role.id=a455148dcf0d4182bf84438cb30422c9, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=739f3b5eaef542c29a0cf0b475c0d965) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:06.561480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ce835-5a43-45e0-be83-47a202abb828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:06.562941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ce835-5a43-45e0-be83-47a202abb828 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:06.568772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 524/1048] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:03:06 2022] GET /identity/v3/role_assignments?user.id=739f3b5eaef542c29a0cf0b475c0d965&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=a455148dcf0d4182bf84438cb30422c9 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:06.610580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70aa0a5-b667-4d56-9d27-9bee732ad824 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:06.649190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d52e7-eadb-4e38-b97e-984a21b85277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:06.650299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d52e7-eadb-4e38-b97e-984a21b85277 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:06.650782 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d52e7-eadb-4e38-b97e-984a21b85277 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:06.651255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d52e7-eadb-4e38-b97e-984a21b85277 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:06.652275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d52e7-eadb-4e38-b97e-984a21b85277 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:06.652789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d52e7-eadb-4e38-b97e-984a21b85277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:06.654552 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d52e7-eadb-4e38-b97e-984a21b85277 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:06.665241 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 525/1049] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:03:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:06.670846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ce835-5a43-45e0-be83-47a202abb828 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:06.779589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ce835-5a43-45e0-be83-47a202abb828 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:06.812374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ac3f4-c4db-43e4-9d48-36046aa50809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:06.813251 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ac3f4-c4db-43e4-9d48-36046aa50809 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:06.813537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ac3f4-c4db-43e4-9d48-36046aa50809 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:06.813774 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ac3f4-c4db-43e4-9d48-36046aa50809 None admin] PATH_INFO: `/v3/users/65c170f06d8d40349ee2e862655fea46` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:06.828398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ac3f4-c4db-43e4-9d48-36046aa50809 None admin] RBAC: Authorizing `identity:get_user(user_id=65c170f06d8d40349ee2e862655fea46)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:06.829986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ac3f4-c4db-43e4-9d48-36046aa50809 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:06.831912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 525/1050] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:06 2022] GET /identity/v3/users/65c170f06d8d40349ee2e862655fea46 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:06.837636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d52e7-eadb-4e38-b97e-984a21b85277 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:06.940980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:06.946339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d52e7-eadb-4e38-b97e-984a21b85277 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:06.983549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea1174-091d-4b8b-89ce-c22645ec5093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:06.984208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea1174-091d-4b8b-89ce-c22645ec5093 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:06.984489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea1174-091d-4b8b-89ce-c22645ec5093 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:06.984756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea1174-091d-4b8b-89ce-c22645ec5093 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:07.053982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea1174-091d-4b8b-89ce-c22645ec5093 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:07.055875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea1174-091d-4b8b-89ce-c22645ec5093 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:07.162175 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:07.169971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 526/1051] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:06 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:07.173203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ac3f4-c4db-43e4-9d48-36046aa50809 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:07.177155 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:07.276536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ac3f4-c4db-43e4-9d48-36046aa50809 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:07.311723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda5958-4eca-455a-a88c-116e0aebcf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:07.312503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda5958-4eca-455a-a88c-116e0aebcf7f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:07.312782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda5958-4eca-455a-a88c-116e0aebcf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:07.313054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda5958-4eca-455a-a88c-116e0aebcf7f None admin] PATH_INFO: `/v3/projects/985f95db5dac42d1808e299aca462b6b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:07.314266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda5958-4eca-455a-a88c-116e0aebcf7f None admin] RBAC: Authorizing `identity:get_project(project_id=985f95db5dac42d1808e299aca462b6b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:07.315698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda5958-4eca-455a-a88c-116e0aebcf7f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:07.317470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 526/1052] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:07 2022] GET /identity/v3/projects/985f95db5dac42d1808e299aca462b6b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:07.323688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea1174-091d-4b8b-89ce-c22645ec5093 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:07.427360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea1174-091d-4b8b-89ce-c22645ec5093 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:07.462982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850441a-374f-4814-b487-73a011579051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:07.463644 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850441a-374f-4814-b487-73a011579051 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:07.463932 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850441a-374f-4814-b487-73a011579051 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:07.464214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850441a-374f-4814-b487-73a011579051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:07.465054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850441a-374f-4814-b487-73a011579051 None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=985f95db5dac42d1808e299aca462b6b, user.id=65c170f06d8d40349ee2e862655fea46) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:07.465370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850441a-374f-4814-b487-73a011579051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:07.466834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850441a-374f-4814-b487-73a011579051 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:07.472768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 527/1053] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:03:07 2022] GET /identity/v3/role_assignments?user.id=65c170f06d8d40349ee2e862655fea46&scope.project.id=985f95db5dac42d1808e299aca462b6b&role.id=2302517080ac42578aa81c606dd0ff2f => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:08.345203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236eae8-99a1-4387-a86f-0e19759a6a8f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:08.345759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236eae8-99a1-4387-a86f-0e19759a6a8f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:08.345759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236eae8-99a1-4387-a86f-0e19759a6a8f None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:08.347385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 527/1054] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:08.352386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38184760-534d-4526-9908-b515362d1010 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:08.352833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38184760-534d-4526-9908-b515362d1010 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:08.353029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38184760-534d-4526-9908-b515362d1010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:08.370755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-38184760-534d-4526-9908-b515362d1010 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:08.371938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-38184760-534d-4526-9908-b515362d1010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:08.431250 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:08.443621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 528/1055] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:08.466242 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236eae8-99a1-4387-a86f-0e19759a6a8f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:08.545342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:08.574562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b2edb-bc47-4bb4-99b9-0e45f94d7ad8 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:08.574893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b2edb-bc47-4bb4-99b9-0e45f94d7ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:08.575157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b2edb-bc47-4bb4-99b9-0e45f94d7ad8 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:08.576540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 529/1056] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:08.579353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236eae8-99a1-4387-a86f-0e19759a6a8f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:08.581495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae519e-60bd-47de-9f96-9760d1b06672 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:08.582091 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae519e-60bd-47de-9f96-9760d1b06672 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:08.582459 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae519e-60bd-47de-9f96-9760d1b06672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:08.599748 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a2ae519e-60bd-47de-9f96-9760d1b06672 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:08.600930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a2ae519e-60bd-47de-9f96-9760d1b06672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:08.617474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbf38b-9811-4326-9aca-90460f0a7b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:08.618299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf38b-9811-4326-9aca-90460f0a7b66 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:08.618585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf38b-9811-4326-9aca-90460f0a7b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:08.618858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf38b-9811-4326-9aca-90460f0a7b66 None admin] PATH_INFO: `/v3/services/image` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:08.619516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbf38b-9811-4326-9aca-90460f0a7b66 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:08.620883 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbf38b-9811-4326-9aca-90460f0a7b66 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9dbbf38b-9811-4326-9aca-90460f0a7b66 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:08.633613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:08.634688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:08.635547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 528/1057] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jul 4 09:03:08 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:08.640621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbf38b-9811-4326-9aca-90460f0a7b66 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:08.673921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 530/1058] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:08.695587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae519e-60bd-47de-9f96-9760d1b06672 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:08.744265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbf38b-9811-4326-9aca-90460f0a7b66 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:08.778560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cafa40-0d7b-48cb-b45e-413e1f3f5736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:08.779232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cafa40-0d7b-48cb-b45e-413e1f3f5736 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:08.779518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cafa40-0d7b-48cb-b45e-413e1f3f5736 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:08.779823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cafa40-0d7b-48cb-b45e-413e1f3f5736 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:08.780518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cafa40-0d7b-48cb-b45e-413e1f3f5736 None admin] RBAC: Adding query filter params (name=image) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:08.780845 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cafa40-0d7b-48cb-b45e-413e1f3f5736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:08.782227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cafa40-0d7b-48cb-b45e-413e1f3f5736 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:08.785030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:08.793366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 529/1059] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:08.794440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cafa40-0d7b-48cb-b45e-413e1f3f5736 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:08.799494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae519e-60bd-47de-9f96-9760d1b06672 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:08.823237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:08.836183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6474336-18bc-4f2b-9197-f6c9f1f0bc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:08.837010 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6474336-18bc-4f2b-9197-f6c9f1f0bc0f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:08.837290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6474336-18bc-4f2b-9197-f6c9f1f0bc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:08.837567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6474336-18bc-4f2b-9197-f6c9f1f0bc0f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:08.848152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6474336-18bc-4f2b-9197-f6c9f1f0bc0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-f6474336-18bc-4f2b-9197-f6c9f1f0bc0f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:08.856460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:08.857582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:08.858534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 531/1060] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 4 09:03:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:03:08.864015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6474336-18bc-4f2b-9197-f6c9f1f0bc0f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:08.907616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cafa40-0d7b-48cb-b45e-413e1f3f5736 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:08.944397 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce37498-bc03-4f15-b5d2-b20e77af61dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:08.945066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce37498-bc03-4f15-b5d2-b20e77af61dd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:08.945350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce37498-bc03-4f15-b5d2-b20e77af61dd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:08.945678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce37498-bc03-4f15-b5d2-b20e77af61dd None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:08.946506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce37498-bc03-4f15-b5d2-b20e77af61dd None admin] RBAC: Adding query filter params (type=image) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:08.946820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce37498-bc03-4f15-b5d2-b20e77af61dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:08.948190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce37498-bc03-4f15-b5d2-b20e77af61dd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:08.953359 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 530/1061] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:08.958952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce37498-bc03-4f15-b5d2-b20e77af61dd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:08.973456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6474336-18bc-4f2b-9197-f6c9f1f0bc0f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:09.009770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf1b77-1a00-4822-996f-773a11de10dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:09.010502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf1b77-1a00-4822-996f-773a11de10dc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:09.010789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf1b77-1a00-4822-996f-773a11de10dc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:09.011111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf1b77-1a00-4822-996f-773a11de10dc None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:09.011868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf1b77-1a00-4822-996f-773a11de10dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:09.012175 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf1b77-1a00-4822-996f-773a11de10dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:09.013542 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf1b77-1a00-4822-996f-773a11de10dc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:09.022296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 532/1062] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 4 09:03:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:09.028261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf1b77-1a00-4822-996f-773a11de10dc None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:09.042631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:09.080099 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:09.083278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce37498-bc03-4f15-b5d2-b20e77af61dd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:09.117556 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d811474-a135-4567-898f-502ae61a6f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:09.118333 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d811474-a135-4567-898f-502ae61a6f9b None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:09.118616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d811474-a135-4567-898f-502ae61a6f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:09.118891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d811474-a135-4567-898f-502ae61a6f9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:09.119635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d811474-a135-4567-898f-502ae61a6f9b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:09.121006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d811474-a135-4567-898f-502ae61a6f9b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:09.141834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf1b77-1a00-4822-996f-773a11de10dc None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:09.143189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 531/1063] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 4 09:03:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:03:09.189516 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b22a2-92c6-4bea-a064-a8acb3b6255a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:09.190361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b22a2-92c6-4bea-a064-a8acb3b6255a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:09.190643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b22a2-92c6-4bea-a064-a8acb3b6255a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:09.190943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b22a2-92c6-4bea-a064-a8acb3b6255a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:09.203808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b22a2-92c6-4bea-a064-a8acb3b6255a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:09.205211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b22a2-92c6-4bea-a064-a8acb3b6255a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-1a2b22a2-92c6-4bea-a064-a8acb3b6255a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:03:09.225044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:09.226107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:09.232131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:03:09.232131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:03:09.232131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:09.232131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:09.232131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 533/1064] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:03:09.232503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d811474-a135-4567-898f-502ae61a6f9b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:09.349733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d811474-a135-4567-898f-502ae61a6f9b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:09.370573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:09.386796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33366e92-0316-4dbd-8f2f-28b887440e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:09.387469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33366e92-0316-4dbd-8f2f-28b887440e64 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:09.387751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33366e92-0316-4dbd-8f2f-28b887440e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:09.388024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33366e92-0316-4dbd-8f2f-28b887440e64 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:09.388753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33366e92-0316-4dbd-8f2f-28b887440e64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:09.389061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33366e92-0316-4dbd-8f2f-28b887440e64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:09.390468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33366e92-0316-4dbd-8f2f-28b887440e64 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:09.402917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 532/1065] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:09.408839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b22a2-92c6-4bea-a064-a8acb3b6255a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:09.447941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad880d-bed7-4486-a29e-29f19a902598 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:09.448301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad880d-bed7-4486-a29e-29f19a902598 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:09.448504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad880d-bed7-4486-a29e-29f19a902598 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:09.450059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 533/1066] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:03:09.455101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cfe22-4409-4240-b730-86c3430add3f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:09.455341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cfe22-4409-4240-b730-86c3430add3f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:09.455624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cfe22-4409-4240-b730-86c3430add3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:09.476177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-121cfe22-4409-4240-b730-86c3430add3f None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:09.477714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-121cfe22-4409-4240-b730-86c3430add3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:09.523126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:09.533420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b22a2-92c6-4bea-a064-a8acb3b6255a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:09.572257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 534/1067] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 04 09:03:09.577327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b2e8-dc92-404d-a350-b99141928a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:09.578170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b2e8-dc92-404d-a350-b99141928a3f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:09.578462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b2e8-dc92-404d-a350-b99141928a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:09.578742 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b2e8-dc92-404d-a350-b99141928a3f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:09.587273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749b2e8-dc92-404d-a350-b99141928a3f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:09.588733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749b2e8-dc92-404d-a350-b99141928a3f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:09.594732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121cfe22-4409-4240-b730-86c3430add3f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-2749b2e8-dc92-404d-a350-b99141928a3f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:03:09.604253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:09.605230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:09.606238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:09.606238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:09.606238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:03:09.606238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:03:09.606238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:03:09.606238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:03:09.606238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:03:09.606238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:03:09.606238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:03:09.606238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:09.610472 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 534/1068] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 4 09:03:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:03:09.611352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b2e8-dc92-404d-a350-b99141928a3f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:09.701866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121cfe22-4409-4240-b730-86c3430add3f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:09.713266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:09.734455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b2e8-dc92-404d-a350-b99141928a3f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:09.739711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb2f30-3257-4f24-a24c-d0a3354ca5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:09.740591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb2f30-3257-4f24-a24c-d0a3354ca5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:09.740942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb2f30-3257-4f24-a24c-d0a3354ca5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:09.741309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb2f30-3257-4f24-a24c-d0a3354ca5d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:09.756053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb2f30-3257-4f24-a24c-d0a3354ca5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2cbb2f30-3257-4f24-a24c-d0a3354ca5d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:09.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:09.767674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:09.768743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:09.773710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 535/1069] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:03:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:09.774589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb2f30-3257-4f24-a24c-d0a3354ca5d2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:09.777744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a64261-6441-453a-a886-ab4852d6cf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:09.778755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a64261-6441-453a-a886-ab4852d6cf4f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:09.779192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a64261-6441-453a-a886-ab4852d6cf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:09.779479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a64261-6441-453a-a886-ab4852d6cf4f None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:09.780330 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a64261-6441-453a-a886-ab4852d6cf4f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:09.780637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a64261-6441-453a-a886-ab4852d6cf4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:09.782102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a64261-6441-453a-a886-ab4852d6cf4f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:09.806445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 535/1070] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 4 09:03:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:09.807379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a64261-6441-453a-a886-ab4852d6cf4f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:09.821709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:09.914569 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb2f30-3257-4f24-a24c-d0a3354ca5d2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:09.926860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a64261-6441-453a-a886-ab4852d6cf4f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:09.952340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2a417-5c39-42f6-9ddd-34c216825f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:09.952994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2a417-5c39-42f6-9ddd-34c216825f0f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:09.953281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2a417-5c39-42f6-9ddd-34c216825f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:09.953575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2a417-5c39-42f6-9ddd-34c216825f0f None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:09.954336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2a417-5c39-42f6-9ddd-34c216825f0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:09.954643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2a417-5c39-42f6-9ddd-34c216825f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:09.956014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2a417-5c39-42f6-9ddd-34c216825f0f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:09.969631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 536/1071] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:03:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:09.969786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4df47-f5f8-4540-949c-792d2c8bbb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:09.969786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4df47-f5f8-4540-949c-792d2c8bbb1a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:09.969786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4df47-f5f8-4540-949c-792d2c8bbb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:09.969786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4df47-f5f8-4540-949c-792d2c8bbb1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:09.970596 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2a417-5c39-42f6-9ddd-34c216825f0f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:09.980080 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4df47-f5f8-4540-949c-792d2c8bbb1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:09.981537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4df47-f5f8-4540-949c-792d2c8bbb1a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-1db4df47-f5f8-4540-949c-792d2c8bbb1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:03:10.000917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:10.001832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:10.002784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:03:10.002784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:03:10.002784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:10.002784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:10.002784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 536/1072] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:10.006673 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:10.008587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4df47-f5f8-4540-949c-792d2c8bbb1a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:10.071462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:10.080993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2a417-5c39-42f6-9ddd-34c216825f0f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:10.115760 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f904bb-0b12-4bd1-91a3-7b59080e06b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:10.116180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4df47-f5f8-4540-949c-792d2c8bbb1a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:10.116631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f904bb-0b12-4bd1-91a3-7b59080e06b1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:10.116909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f904bb-0b12-4bd1-91a3-7b59080e06b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:10.117178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f904bb-0b12-4bd1-91a3-7b59080e06b1 None admin] PATH_INFO: `/v3/users/65c170f06d8d40349ee2e862655fea46` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:10.118385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f904bb-0b12-4bd1-91a3-7b59080e06b1 None admin] RBAC: Authorizing `identity:get_user(user_id=65c170f06d8d40349ee2e862655fea46)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:10.119824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f904bb-0b12-4bd1-91a3-7b59080e06b1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:10.121607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 537/1073] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:09 2022] GET /identity/v3/users/65c170f06d8d40349ee2e862655fea46 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:10.127151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f904bb-0b12-4bd1-91a3-7b59080e06b1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:10.165388 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eacdf4-6093-42c6-b15d-ec2c7150891c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:10.166110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eacdf4-6093-42c6-b15d-ec2c7150891c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:10.166399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eacdf4-6093-42c6-b15d-ec2c7150891c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:10.166677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eacdf4-6093-42c6-b15d-ec2c7150891c None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:10.167408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eacdf4-6093-42c6-b15d-ec2c7150891c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:10.167715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eacdf4-6093-42c6-b15d-ec2c7150891c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:10.169496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eacdf4-6093-42c6-b15d-ec2c7150891c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:10.181420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 537/1074] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:10.187248 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eacdf4-6093-42c6-b15d-ec2c7150891c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:10.229488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:10.232992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f904bb-0b12-4bd1-91a3-7b59080e06b1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:10.267717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34d28f-dc0f-4f9a-bdfc-2a932477d4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:10.268580 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34d28f-dc0f-4f9a-bdfc-2a932477d4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:10.268859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34d28f-dc0f-4f9a-bdfc-2a932477d4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:10.269160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34d28f-dc0f-4f9a-bdfc-2a932477d4c8 None admin] PATH_INFO: `/v3/projects/985f95db5dac42d1808e299aca462b6b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:10.270367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34d28f-dc0f-4f9a-bdfc-2a932477d4c8 None admin] RBAC: Authorizing `identity:get_project(project_id=985f95db5dac42d1808e299aca462b6b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:10.271804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34d28f-dc0f-4f9a-bdfc-2a932477d4c8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:10.273689 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 538/1075] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:10 2022] GET /identity/v3/projects/985f95db5dac42d1808e299aca462b6b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:10.279196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34d28f-dc0f-4f9a-bdfc-2a932477d4c8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:10.294019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eacdf4-6093-42c6-b15d-ec2c7150891c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:10.318573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:10.331330 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc17e0d-b9df-4723-bf2e-08263667d9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:10.332173 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc17e0d-b9df-4723-bf2e-08263667d9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:10.332454 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc17e0d-b9df-4723-bf2e-08263667d9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:10.332740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc17e0d-b9df-4723-bf2e-08263667d9f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:10.340219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc17e0d-b9df-4723-bf2e-08263667d9f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:10.341788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc17e0d-b9df-4723-bf2e-08263667d9f8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-2bc17e0d-b9df-4723-bf2e-08263667d9f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:03:10.356426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:10.357506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:10.358619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:03:10.358619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:03:10.358619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:03:10.358619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:03:10.358619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:03:10.358619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:03:10.358619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:03:10.358619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:10.358619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 538/1076] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:03:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:10.363451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc17e0d-b9df-4723-bf2e-08263667d9f8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:10.390391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34d28f-dc0f-4f9a-bdfc-2a932477d4c8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:10.426563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b77c-a5f3-43a3-a9fd-b821966964f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:10.427457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b77c-a5f3-43a3-a9fd-b821966964f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:10.427739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b77c-a5f3-43a3-a9fd-b821966964f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:10.428014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b77c-a5f3-43a3-a9fd-b821966964f1 None admin] PATH_INFO: `/v3/projects/985f95db5dac42d1808e299aca462b6b/users/65c170f06d8d40349ee2e862655fea46/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:10.430123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30b77c-a5f3-43a3-a9fd-b821966964f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=985f95db5dac42d1808e299aca462b6b, user_id=65c170f06d8d40349ee2e862655fea46, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:10.431697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30b77c-a5f3-43a3-a9fd-b821966964f1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:10.445727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 539/1077] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:03:10 2022] PUT /identity/v3/projects/985f95db5dac42d1808e299aca462b6b/users/65c170f06d8d40349ee2e862655fea46/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:03:10.478804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc17e0d-b9df-4723-bf2e-08263667d9f8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:10.518698 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b27e42-2795-43d2-8945-e244f4bc02b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:10.519416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b27e42-2795-43d2-8945-e244f4bc02b3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:10.519702 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b27e42-2795-43d2-8945-e244f4bc02b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:10.519981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b27e42-2795-43d2-8945-e244f4bc02b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:10.520842 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b27e42-2795-43d2-8945-e244f4bc02b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:10.521160 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b27e42-2795-43d2-8945-e244f4bc02b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:10.522663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b27e42-2795-43d2-8945-e244f4bc02b3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:10.536099 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 539/1078] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:03:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:10.542247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b77c-a5f3-43a3-a9fd-b821966964f1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:10.574793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:10.656155 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b77c-a5f3-43a3-a9fd-b821966964f1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:10.692116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4eda6a-1de9-4c5f-baed-3bd8a7d76a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:10.693015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eda6a-1de9-4c5f-baed-3bd8a7d76a5b None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:10.693288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eda6a-1de9-4c5f-baed-3bd8a7d76a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:10.693564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eda6a-1de9-4c5f-baed-3bd8a7d76a5b None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/739f3b5eaef542c29a0cf0b475c0d965/roles/a455148dcf0d4182bf84438cb30422c9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:10.695860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4eda6a-1de9-4c5f-baed-3bd8a7d76a5b None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=739f3b5eaef542c29a0cf0b475c0d965, role_id=a455148dcf0d4182bf84438cb30422c9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:10.697305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4eda6a-1de9-4c5f-baed-3bd8a7d76a5b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:10.712488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 540/1079] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:03:10 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/739f3b5eaef542c29a0cf0b475c0d965/roles/a455148dcf0d4182bf84438cb30422c9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:03:11.218105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441f2fc-a062-437f-af61-c18d31f9abc6 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:11.218456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441f2fc-a062-437f-af61-c18d31f9abc6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:11.218626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441f2fc-a062-437f-af61-c18d31f9abc6 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:11.220199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 540/1080] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:11.225484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19a67-c539-47ec-8fcf-41a362e1328a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:11.225661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19a67-c539-47ec-8fcf-41a362e1328a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:11.226004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19a67-c539-47ec-8fcf-41a362e1328a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:11.243440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-09b19a67-c539-47ec-8fcf-41a362e1328a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:11.244622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-09b19a67-c539-47ec-8fcf-41a362e1328a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:11.314999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 541/1081] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:11.337302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441f2fc-a062-437f-af61-c18d31f9abc6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:11.384875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:11.446857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441f2fc-a062-437f-af61-c18d31f9abc6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:11.482231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca7e0d-0473-4041-a4a5-d333ea043709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:11.482878 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca7e0d-0473-4041-a4a5-d333ea043709 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:11.483152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca7e0d-0473-4041-a4a5-d333ea043709 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:11.483432 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca7e0d-0473-4041-a4a5-d333ea043709 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:11.553716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca7e0d-0473-4041-a4a5-d333ea043709 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:11.555261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca7e0d-0473-4041-a4a5-d333ea043709 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:11.597032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:11.655684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 541/1082] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:11.661491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19a67-c539-47ec-8fcf-41a362e1328a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:11.680317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:11.763160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19a67-c539-47ec-8fcf-41a362e1328a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:11.799768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba238b1c-e55b-47ef-986b-aa553999654e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:11.800712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba238b1c-e55b-47ef-986b-aa553999654e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:11.800990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba238b1c-e55b-47ef-986b-aa553999654e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:11.801280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba238b1c-e55b-47ef-986b-aa553999654e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:11.812043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba238b1c-e55b-47ef-986b-aa553999654e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:11.813499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba238b1c-e55b-47ef-986b-aa553999654e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ba238b1c-e55b-47ef-986b-aa553999654e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:03:11.830982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:11.831982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:11.832905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:03:11.832905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:03:11.832905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:11.832905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:11.834254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 542/1083] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:11.838298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca7e0d-0473-4041-a4a5-d333ea043709 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:11.940485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca7e0d-0473-4041-a4a5-d333ea043709 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:11.975021 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb2bae-4537-4529-8583-81893f1d526a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:11.975684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb2bae-4537-4529-8583-81893f1d526a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:11.975970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb2bae-4537-4529-8583-81893f1d526a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:11.976241 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb2bae-4537-4529-8583-81893f1d526a None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:11.976970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb2bae-4537-4529-8583-81893f1d526a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:11.977280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb2bae-4537-4529-8583-81893f1d526a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:11.978685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb2bae-4537-4529-8583-81893f1d526a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:11.990215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 542/1084] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:12.463991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e920c0e-9613-4596-a607-75ef3906c075 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:12.464257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e920c0e-9613-4596-a607-75ef3906c075 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:12.464448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e920c0e-9613-4596-a607-75ef3906c075 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:12.465824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 543/1085] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:12.471368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ccbd4-3b70-4c7e-aace-d29c9a1c8c5e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:12.471660 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ccbd4-3b70-4c7e-aace-d29c9a1c8c5e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:12.471935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ccbd4-3b70-4c7e-aace-d29c9a1c8c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:12.477740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:12.491267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-536ccbd4-3b70-4c7e-aace-d29c9a1c8c5e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:12.492447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-536ccbd4-3b70-4c7e-aace-d29c9a1c8c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:12.561196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 543/1086] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:12.584876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e920c0e-9613-4596-a607-75ef3906c075 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:12.662464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:12.696098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e920c0e-9613-4596-a607-75ef3906c075 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:12.707296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a1b6e-7d27-49d4-89c5-aaaf82fdaebf None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:12.707657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a1b6e-7d27-49d4-89c5-aaaf82fdaebf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:12.707845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a1b6e-7d27-49d4-89c5-aaaf82fdaebf None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:12.709212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 544/1087] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:03:12.715529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301158a-b942-4389-9346-c787ac8efb1e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:12.715762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301158a-b942-4389-9346-c787ac8efb1e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:12.716063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301158a-b942-4389-9346-c787ac8efb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:12.735147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3301158a-b942-4389-9346-c787ac8efb1e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:12.735424 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6725b-6884-404c-851c-81ea2058e5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:12.736158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6725b-6884-404c-851c-81ea2058e5ca None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:12.736449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6725b-6884-404c-851c-81ea2058e5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:12.736661 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3301158a-b942-4389-9346-c787ac8efb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:12.736769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6725b-6884-404c-851c-81ea2058e5ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:12.747994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6725b-6884-404c-851c-81ea2058e5ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a3b6725b-6884-404c-851c-81ea2058e5ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:12.756851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:12.757809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:12.758793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 544/1088] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:03:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:12.764091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6725b-6884-404c-851c-81ea2058e5ca None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:12.818448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 545/1089] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:12.842625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301158a-b942-4389-9346-c787ac8efb1e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:12.885673 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6725b-6884-404c-851c-81ea2058e5ca None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:12.918364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:12.921797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:12.929555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791c85b-7002-4c8a-820a-a38ad2a67527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:12.930167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791c85b-7002-4c8a-820a-a38ad2a67527 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:12.930449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791c85b-7002-4c8a-820a-a38ad2a67527 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:12.930731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791c85b-7002-4c8a-820a-a38ad2a67527 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:12.931490 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791c85b-7002-4c8a-820a-a38ad2a67527 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:12.931805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791c85b-7002-4c8a-820a-a38ad2a67527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:12.933194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791c85b-7002-4c8a-820a-a38ad2a67527 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:12.942321 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 545/1090] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:03:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:12.948973 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791c85b-7002-4c8a-820a-a38ad2a67527 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:12.973663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301158a-b942-4389-9346-c787ac8efb1e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:13.019315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5ee9f-99cc-45a5-9bb4-a36f3b8c7e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:13.019916 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ee9f-99cc-45a5-9bb4-a36f3b8c7e61 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:13.020209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ee9f-99cc-45a5-9bb4-a36f3b8c7e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:13.020465 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ee9f-99cc-45a5-9bb4-a36f3b8c7e61 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:13.033556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5ee9f-99cc-45a5-9bb4-a36f3b8c7e61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-59d5ee9f-99cc-45a5-9bb4-a36f3b8c7e61 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:13.043832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:13.045017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:13.045911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 546/1091] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 4 09:03:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:13.051831 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5ee9f-99cc-45a5-9bb4-a36f3b8c7e61 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:13.092007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791c85b-7002-4c8a-820a-a38ad2a67527 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:13.129923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed3d8c-d66d-4a23-aaa5-e1fa751467f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:13.131113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed3d8c-d66d-4a23-aaa5-e1fa751467f7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:13.131397 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed3d8c-d66d-4a23-aaa5-e1fa751467f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:13.131877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed3d8c-d66d-4a23-aaa5-e1fa751467f7 None admin] PATH_INFO: `/v3/users/65c170f06d8d40349ee2e862655fea46` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:13.133366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed3d8c-d66d-4a23-aaa5-e1fa751467f7 None admin] RBAC: Authorizing `identity:get_user(user_id=65c170f06d8d40349ee2e862655fea46)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:13.134861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed3d8c-d66d-4a23-aaa5-e1fa751467f7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:13.136507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 546/1092] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:12 2022] GET /identity/v3/users/65c170f06d8d40349ee2e862655fea46 => generated 285 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:13.142140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed3d8c-d66d-4a23-aaa5-e1fa751467f7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:13.178919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5ee9f-99cc-45a5-9bb4-a36f3b8c7e61 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:13.207513 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:13.209434 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:13.220869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3248da-dfb5-4b2f-9fcf-eb0e8c2b5a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:13.221534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3248da-dfb5-4b2f-9fcf-eb0e8c2b5a25 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:13.221837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3248da-dfb5-4b2f-9fcf-eb0e8c2b5a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:13.222148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3248da-dfb5-4b2f-9fcf-eb0e8c2b5a25 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:13.223318 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3248da-dfb5-4b2f-9fcf-eb0e8c2b5a25 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:13.223635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3248da-dfb5-4b2f-9fcf-eb0e8c2b5a25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:13.225081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3248da-dfb5-4b2f-9fcf-eb0e8c2b5a25 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:13.233679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 547/1093] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 4 09:03:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:13.240262 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3248da-dfb5-4b2f-9fcf-eb0e8c2b5a25 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:13.256495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed3d8c-d66d-4a23-aaa5-e1fa751467f7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:13.293051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b27ff-2b93-4444-8780-5fb2f8369e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:13.293829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b27ff-2b93-4444-8780-5fb2f8369e1d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:13.294259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b27ff-2b93-4444-8780-5fb2f8369e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:13.295399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b27ff-2b93-4444-8780-5fb2f8369e1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:13.365413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3248da-dfb5-4b2f-9fcf-eb0e8c2b5a25 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:13.379401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b27ff-2b93-4444-8780-5fb2f8369e1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:13.381364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b27ff-2b93-4444-8780-5fb2f8369e1d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:13.404388 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb25de4-e2f0-436f-939f-b37ac13ba8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:13.405177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb25de4-e2f0-436f-939f-b37ac13ba8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:13.405457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb25de4-e2f0-436f-939f-b37ac13ba8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:13.405730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb25de4-e2f0-436f-939f-b37ac13ba8d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:13.418909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb25de4-e2f0-436f-939f-b37ac13ba8d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:13.420715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb25de4-e2f0-436f-939f-b37ac13ba8d6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-0bb25de4-e2f0-436f-939f-b37ac13ba8d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:03:13.442705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:13.444345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:13.445860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:03:13.445860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:03:13.445860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:13.445860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:13.445860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 548/1094] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:13.449719 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb25de4-e2f0-436f-939f-b37ac13ba8d6 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:13.452666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:13.487927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:13.502105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 547/1095] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:13.504748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b27ff-2b93-4444-8780-5fb2f8369e1d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:13.559204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb25de4-e2f0-436f-939f-b37ac13ba8d6 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:13.593905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431dbff4-9ffa-42f2-b934-5f39fc7cea58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:13.594579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dbff4-9ffa-42f2-b934-5f39fc7cea58 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:13.594863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dbff4-9ffa-42f2-b934-5f39fc7cea58 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:13.595137 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dbff4-9ffa-42f2-b934-5f39fc7cea58 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:13.595863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431dbff4-9ffa-42f2-b934-5f39fc7cea58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:13.596171 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431dbff4-9ffa-42f2-b934-5f39fc7cea58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:13.597584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431dbff4-9ffa-42f2-b934-5f39fc7cea58 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:13.605455 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b27ff-2b93-4444-8780-5fb2f8369e1d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:13.608811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 549/1096] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:13.614499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431dbff4-9ffa-42f2-b934-5f39fc7cea58 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:13.639321 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb9854-aac1-4963-b1b4-9b2371b04877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:13.640186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb9854-aac1-4963-b1b4-9b2371b04877 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:13.640473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb9854-aac1-4963-b1b4-9b2371b04877 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:13.640754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb9854-aac1-4963-b1b4-9b2371b04877 None admin] PATH_INFO: `/v3/projects/985f95db5dac42d1808e299aca462b6b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:13.641970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb9854-aac1-4963-b1b4-9b2371b04877 None admin] RBAC: Authorizing `identity:get_project(project_id=985f95db5dac42d1808e299aca462b6b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:13.643411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb9854-aac1-4963-b1b4-9b2371b04877 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:13.645179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 548/1097] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:13 2022] GET /identity/v3/projects/985f95db5dac42d1808e299aca462b6b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:13.651735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb9854-aac1-4963-b1b4-9b2371b04877 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:13.679822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:13.708924 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:13.715140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431dbff4-9ffa-42f2-b934-5f39fc7cea58 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:13.750962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e0c9e-91bc-4c8b-889d-e287820b9e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:13.751679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e0c9e-91bc-4c8b-889d-e287820b9e7f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:13.751962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e0c9e-91bc-4c8b-889d-e287820b9e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:13.752070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb9854-aac1-4963-b1b4-9b2371b04877 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:13.752239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e0c9e-91bc-4c8b-889d-e287820b9e7f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:13.760435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e0c9e-91bc-4c8b-889d-e287820b9e7f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:13.761908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e0c9e-91bc-4c8b-889d-e287820b9e7f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-627e0c9e-91bc-4c8b-889d-e287820b9e7f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:03:13.776239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:13.777280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:13.778293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:13.778293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:13.778293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:03:13.778293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:03:13.778293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:03:13.778293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:03:13.778293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:03:13.778293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:03:13.778293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 04 09:03:13.778293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:13.778293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 550/1098] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 4 09:03:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:03:13.783369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e0c9e-91bc-4c8b-889d-e287820b9e7f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:13.790149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc5332-0854-469f-9c12-e244254a41c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:13.790814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc5332-0854-469f-9c12-e244254a41c4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:13.791096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc5332-0854-469f-9c12-e244254a41c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:13.791368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc5332-0854-469f-9c12-e244254a41c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:13.792190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc5332-0854-469f-9c12-e244254a41c4 None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=985f95db5dac42d1808e299aca462b6b, user.id=65c170f06d8d40349ee2e862655fea46) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:13.792498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc5332-0854-469f-9c12-e244254a41c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:13.793918 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc5332-0854-469f-9c12-e244254a41c4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:13.799690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 549/1099] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:03:13 2022] GET /identity/v3/role_assignments?user.id=65c170f06d8d40349ee2e862655fea46&scope.project.id=985f95db5dac42d1808e299aca462b6b&role.id=2302517080ac42578aa81c606dd0ff2f => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:13.897002 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e0c9e-91bc-4c8b-889d-e287820b9e7f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:13.935142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171d782-6229-44a4-b1aa-4ac86e419d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:13.935862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171d782-6229-44a4-b1aa-4ac86e419d7c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:13.936146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171d782-6229-44a4-b1aa-4ac86e419d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:13.936422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171d782-6229-44a4-b1aa-4ac86e419d7c None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:13.937291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171d782-6229-44a4-b1aa-4ac86e419d7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:13.937604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171d782-6229-44a4-b1aa-4ac86e419d7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:13.939109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171d782-6229-44a4-b1aa-4ac86e419d7c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:13.959156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 551/1100] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 4 09:03:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:13.965124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc5332-0854-469f-9c12-e244254a41c4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:14.061832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:14.071134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc5332-0854-469f-9c12-e244254a41c4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:14.106522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb3d7e-d9c0-42d2-a7ba-8dece490ebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:14.107174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb3d7e-d9c0-42d2-a7ba-8dece490ebbb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:14.107456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb3d7e-d9c0-42d2-a7ba-8dece490ebbb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:14.107733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb3d7e-d9c0-42d2-a7ba-8dece490ebbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:14.179142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb3d7e-d9c0-42d2-a7ba-8dece490ebbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:14.180612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb3d7e-d9c0-42d2-a7ba-8dece490ebbb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:14.271130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:14.282344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 550/1101] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:14.288370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171d782-6229-44a4-b1aa-4ac86e419d7c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:14.304418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:14.398773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171d782-6229-44a4-b1aa-4ac86e419d7c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:14.434133 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120fec4-f0bc-4333-b351-74f285360fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:14.434920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120fec4-f0bc-4333-b351-74f285360fdd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:14.435204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120fec4-f0bc-4333-b351-74f285360fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:14.435476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120fec4-f0bc-4333-b351-74f285360fdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:14.445720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120fec4-f0bc-4333-b351-74f285360fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:14.447199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120fec4-f0bc-4333-b351-74f285360fdd None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5120fec4-f0bc-4333-b351-74f285360fdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:03:14.464423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:14.465368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:14.466359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:03:14.466359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:03:14.466359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:03:14.466359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:14.466359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 552/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:14.471529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb3d7e-d9c0-42d2-a7ba-8dece490ebbb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:14.576470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb3d7e-d9c0-42d2-a7ba-8dece490ebbb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:14.611493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9d10f-72d0-4a71-97c0-4e2f21afdd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:14.612152 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9d10f-72d0-4a71-97c0-4e2f21afdd18 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:14.612436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9d10f-72d0-4a71-97c0-4e2f21afdd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:14.612710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9d10f-72d0-4a71-97c0-4e2f21afdd18 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:14.613732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9d10f-72d0-4a71-97c0-4e2f21afdd18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:14.614076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9d10f-72d0-4a71-97c0-4e2f21afdd18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:14.615447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9d10f-72d0-4a71-97c0-4e2f21afdd18 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:14.626740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 551/1103] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:14.633101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120fec4-f0bc-4333-b351-74f285360fdd None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:14.705453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:14.741670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120fec4-f0bc-4333-b351-74f285360fdd None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:14.776556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0295e0-a856-4122-b936-a8b45873f9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:14.777340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0295e0-a856-4122-b936-a8b45873f9bf None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:14.777624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0295e0-a856-4122-b936-a8b45873f9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:14.777924 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0295e0-a856-4122-b936-a8b45873f9bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:14.784833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0295e0-a856-4122-b936-a8b45873f9bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:14.786319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0295e0-a856-4122-b936-a8b45873f9bf None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ed0295e0-a856-4122-b936-a8b45873f9bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:03:14.799993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:14.800981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:14.802103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:03:14.802103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:03:14.802103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:03:14.802103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:03:14.802103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:03:14.802103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:03:14.802103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:03:14.802103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:14.802103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 553/1104] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:03:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:14.807150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9d10f-72d0-4a71-97c0-4e2f21afdd18 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:14.862179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:14.911237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9d10f-72d0-4a71-97c0-4e2f21afdd18 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:14.948075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40858802-eba9-4b42-a914-7403b3f58804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:14.948740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40858802-eba9-4b42-a914-7403b3f58804 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:14.949023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40858802-eba9-4b42-a914-7403b3f58804 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:14.949302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40858802-eba9-4b42-a914-7403b3f58804 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:14.950134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40858802-eba9-4b42-a914-7403b3f58804 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:14.950446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40858802-eba9-4b42-a914-7403b3f58804 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:14.951878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40858802-eba9-4b42-a914-7403b3f58804 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:14.964377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 552/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:03:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:14.970463 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0295e0-a856-4122-b936-a8b45873f9bf None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:15.074598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0295e0-a856-4122-b936-a8b45873f9bf None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:15.100388 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:15.109969 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e44948-b11e-4862-9616-882837bd036b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:15.110634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44948-b11e-4862-9616-882837bd036b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:15.110920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44948-b11e-4862-9616-882837bd036b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:15.111199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44948-b11e-4862-9616-882837bd036b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:15.112029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e44948-b11e-4862-9616-882837bd036b None admin] RBAC: Adding query filter params (role.id=a455148dcf0d4182bf84438cb30422c9, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=739f3b5eaef542c29a0cf0b475c0d965) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:15.112339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e44948-b11e-4862-9616-882837bd036b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:15.114123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e44948-b11e-4862-9616-882837bd036b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:15.120315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 554/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:03:14 2022] GET /identity/v3/role_assignments?user.id=739f3b5eaef542c29a0cf0b475c0d965&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=a455148dcf0d4182bf84438cb30422c9 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:15.840559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0a3e4-df62-4524-a2c8-e9263a837129 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:15.841107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0a3e4-df62-4524-a2c8-e9263a837129 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:15.841107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0a3e4-df62-4524-a2c8-e9263a837129 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:15.842607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 553/1107] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:15.847897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964eb5-2c96-40a4-877b-79789d6798dc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:15.848389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964eb5-2c96-40a4-877b-79789d6798dc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:15.848479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964eb5-2c96-40a4-877b-79789d6798dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:15.867774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c4964eb5-2c96-40a4-877b-79789d6798dc None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:15.868973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c4964eb5-2c96-40a4-877b-79789d6798dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:15.939951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 555/1108] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:15.964496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0a3e4-df62-4524-a2c8-e9263a837129 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:16.069823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0a3e4-df62-4524-a2c8-e9263a837129 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:16.082964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:16.117420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140f9ba-f3a3-41e2-b2e1-5a6a2a8762f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:16.118169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140f9ba-f3a3-41e2-b2e1-5a6a2a8762f2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:16.118455 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140f9ba-f3a3-41e2-b2e1-5a6a2a8762f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:16.118724 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140f9ba-f3a3-41e2-b2e1-5a6a2a8762f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:16.120020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140f9ba-f3a3-41e2-b2e1-5a6a2a8762f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:16.121457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140f9ba-f3a3-41e2-b2e1-5a6a2a8762f2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:16.123104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 554/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:16.129607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964eb5-2c96-40a4-877b-79789d6798dc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:16.238079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964eb5-2c96-40a4-877b-79789d6798dc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:16.265129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:16.280589 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6130b4-9060-42ce-9f7e-5e3d6d91ad44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:16.281255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6130b4-9060-42ce-9f7e-5e3d6d91ad44 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:16.281534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6130b4-9060-42ce-9f7e-5e3d6d91ad44 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:16.281802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6130b4-9060-42ce-9f7e-5e3d6d91ad44 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:16.282533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6130b4-9060-42ce-9f7e-5e3d6d91ad44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:16.283961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6130b4-9060-42ce-9f7e-5e3d6d91ad44 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:16.305257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 556/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:03:16 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:03:18.327693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bae46-1cf3-458d-bd11-18d9cb9d3c0b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:18.328548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bae46-1cf3-458d-bd11-18d9cb9d3c0b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:18.328548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bae46-1cf3-458d-bd11-18d9cb9d3c0b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:18.330869 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 555/1111] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:18.336386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec8e5c-88aa-48e0-a678-b2bb2c60fac4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:18.336813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec8e5c-88aa-48e0-a678-b2bb2c60fac4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:18.336905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec8e5c-88aa-48e0-a678-b2bb2c60fac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:18.354960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f1ec8e5c-88aa-48e0-a678-b2bb2c60fac4 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:18.356153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f1ec8e5c-88aa-48e0-a678-b2bb2c60fac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:18.433295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 557/1112] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:18.460288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bae46-1cf3-458d-bd11-18d9cb9d3c0b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:18.572499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bae46-1cf3-458d-bd11-18d9cb9d3c0b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:18.613532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f385ffc-9aee-4d27-a5fe-9eb0eb339ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:18.614825 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f385ffc-9aee-4d27-a5fe-9eb0eb339ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:18.615183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f385ffc-9aee-4d27-a5fe-9eb0eb339ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:18.615554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f385ffc-9aee-4d27-a5fe-9eb0eb339ee5 None admin] PATH_INFO: `/v3/roles/2eea147ab8b1426294a08106410f6d45` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:18.617767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f385ffc-9aee-4d27-a5fe-9eb0eb339ee5 None admin] RBAC: Authorizing `identity:get_role(role_id=2eea147ab8b1426294a08106410f6d45)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:18.619226 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f385ffc-9aee-4d27-a5fe-9eb0eb339ee5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:18.620603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 556/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:18 2022] GET /identity/v3/roles/2eea147ab8b1426294a08106410f6d45 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:18.626791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec8e5c-88aa-48e0-a678-b2bb2c60fac4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:18.735767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec8e5c-88aa-48e0-a678-b2bb2c60fac4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:18.743511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:18.771700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091c712-4435-4a96-a530-e20b6b3c0861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:18.772578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091c712-4435-4a96-a530-e20b6b3c0861 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:18.772863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091c712-4435-4a96-a530-e20b6b3c0861 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:18.773143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091c712-4435-4a96-a530-e20b6b3c0861 None admin] PATH_INFO: `/v3/users/a678fa6854274fcf84086a7a0ee0f202` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:18.786153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091c712-4435-4a96-a530-e20b6b3c0861 None admin] RBAC: Authorizing `identity:get_user(user_id=a678fa6854274fcf84086a7a0ee0f202)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:18.787603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091c712-4435-4a96-a530-e20b6b3c0861 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:18.789229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 558/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:18 2022] GET /identity/v3/users/a678fa6854274fcf84086a7a0ee0f202 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:18.795156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f385ffc-9aee-4d27-a5fe-9eb0eb339ee5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:18.843579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:18.898349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f385ffc-9aee-4d27-a5fe-9eb0eb339ee5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:18.934107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054bbd4-3da9-44ef-8283-310980f2087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:18.934751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054bbd4-3da9-44ef-8283-310980f2087c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:18.935035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054bbd4-3da9-44ef-8283-310980f2087c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:18.935308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054bbd4-3da9-44ef-8283-310980f2087c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:19.004571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054bbd4-3da9-44ef-8283-310980f2087c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:19.005995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054bbd4-3da9-44ef-8283-310980f2087c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:19.053666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:19.106273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 557/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:19.112523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091c712-4435-4a96-a530-e20b6b3c0861 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:19.218968 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091c712-4435-4a96-a530-e20b6b3c0861 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:19.254701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad1b4e-5263-4272-bee1-393cba4f03f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:19.255488 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad1b4e-5263-4272-bee1-393cba4f03f2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:19.255769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad1b4e-5263-4272-bee1-393cba4f03f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:19.256042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad1b4e-5263-4272-bee1-393cba4f03f2 None admin] PATH_INFO: `/v3/projects/985f95db5dac42d1808e299aca462b6b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:19.257273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad1b4e-5263-4272-bee1-393cba4f03f2 None admin] RBAC: Authorizing `identity:get_project(project_id=985f95db5dac42d1808e299aca462b6b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:19.258740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad1b4e-5263-4272-bee1-393cba4f03f2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:19.260422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 559/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:19 2022] GET /identity/v3/projects/985f95db5dac42d1808e299aca462b6b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:19.266879 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054bbd4-3da9-44ef-8283-310980f2087c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:19.371378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054bbd4-3da9-44ef-8283-310980f2087c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:19.411959 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69193e62-adbf-469b-b413-b005f15e666b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:19.412613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69193e62-adbf-469b-b413-b005f15e666b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:19.412897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69193e62-adbf-469b-b413-b005f15e666b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:19.413176 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69193e62-adbf-469b-b413-b005f15e666b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:19.414022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69193e62-adbf-469b-b413-b005f15e666b None admin] RBAC: Adding query filter params (role.id=2eea147ab8b1426294a08106410f6d45, scope.project.id=985f95db5dac42d1808e299aca462b6b, user.id=a678fa6854274fcf84086a7a0ee0f202) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:19.414337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69193e62-adbf-469b-b413-b005f15e666b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:19.415758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69193e62-adbf-469b-b413-b005f15e666b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:19.421314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 558/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:03:19 2022] GET /identity/v3/role_assignments?user.id=a678fa6854274fcf84086a7a0ee0f202&scope.project.id=985f95db5dac42d1808e299aca462b6b&role.id=2eea147ab8b1426294a08106410f6d45 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:21.402720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeca41-7d40-43cb-a05a-05899ee0aa45 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:21.403565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeca41-7d40-43cb-a05a-05899ee0aa45 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:21.403565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeca41-7d40-43cb-a05a-05899ee0aa45 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:21.405859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 560/1118] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:21.411496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a599895-55bf-477e-b419-ff5568bb544c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:21.411952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a599895-55bf-477e-b419-ff5568bb544c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:21.412115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a599895-55bf-477e-b419-ff5568bb544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:21.430129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4a599895-55bf-477e-b419-ff5568bb544c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:21.431357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4a599895-55bf-477e-b419-ff5568bb544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:21.438767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:21.501091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 559/1119] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:21.524606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aeca41-7d40-43cb-a05a-05899ee0aa45 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:21.563820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:21.639770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aeca41-7d40-43cb-a05a-05899ee0aa45 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:21.675206 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a7596-7ccf-4878-89ac-6ceee28a6ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:21.676043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a7596-7ccf-4878-89ac-6ceee28a6ace None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:21.676319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a7596-7ccf-4878-89ac-6ceee28a6ace None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:21.676585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a7596-7ccf-4878-89ac-6ceee28a6ace None admin] PATH_INFO: `/v3/roles/2eea147ab8b1426294a08106410f6d45` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:21.678197 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a7596-7ccf-4878-89ac-6ceee28a6ace None admin] RBAC: Authorizing `identity:get_role(role_id=2eea147ab8b1426294a08106410f6d45)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:21.679560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a7596-7ccf-4878-89ac-6ceee28a6ace None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:21.680899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 561/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:21 2022] GET /identity/v3/roles/2eea147ab8b1426294a08106410f6d45 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:21.690306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a599895-55bf-477e-b419-ff5568bb544c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:21.795199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a599895-55bf-477e-b419-ff5568bb544c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:21.832183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5331b7-7ed5-48d0-bba1-5ad88cdb7d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:21.833098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5331b7-7ed5-48d0-bba1-5ad88cdb7d42 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:21.833376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5331b7-7ed5-48d0-bba1-5ad88cdb7d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:21.833648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5331b7-7ed5-48d0-bba1-5ad88cdb7d42 None admin] PATH_INFO: `/v3/users/a678fa6854274fcf84086a7a0ee0f202` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:21.834998 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5331b7-7ed5-48d0-bba1-5ad88cdb7d42 None admin] RBAC: Authorizing `identity:get_user(user_id=a678fa6854274fcf84086a7a0ee0f202)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:21.836426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5331b7-7ed5-48d0-bba1-5ad88cdb7d42 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:21.838066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 560/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:21 2022] GET /identity/v3/users/a678fa6854274fcf84086a7a0ee0f202 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:21.847346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a7596-7ccf-4878-89ac-6ceee28a6ace None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:21.951806 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:21.954861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a7596-7ccf-4878-89ac-6ceee28a6ace None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:21.993681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fcf6ac-8c1b-4194-9218-b967fdf06564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:21.994579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcf6ac-8c1b-4194-9218-b967fdf06564 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:21.994866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcf6ac-8c1b-4194-9218-b967fdf06564 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:21.995148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcf6ac-8c1b-4194-9218-b967fdf06564 None admin] PATH_INFO: `/v3/projects/985f95db5dac42d1808e299aca462b6b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:21.996406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fcf6ac-8c1b-4194-9218-b967fdf06564 None admin] RBAC: Authorizing `identity:get_project(project_id=985f95db5dac42d1808e299aca462b6b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:21.997857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fcf6ac-8c1b-4194-9218-b967fdf06564 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:21.999563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 562/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:21 2022] GET /identity/v3/projects/985f95db5dac42d1808e299aca462b6b => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:22.006591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5331b7-7ed5-48d0-bba1-5ad88cdb7d42 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:22.019548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:22.115077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5331b7-7ed5-48d0-bba1-5ad88cdb7d42 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:22.151620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e45b8-5d30-4122-9b0a-a035cc8922e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:22.152519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e45b8-5d30-4122-9b0a-a035cc8922e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:22.152796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e45b8-5d30-4122-9b0a-a035cc8922e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:22.153069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e45b8-5d30-4122-9b0a-a035cc8922e3 None admin] PATH_INFO: `/v3/projects/985f95db5dac42d1808e299aca462b6b/users/a678fa6854274fcf84086a7a0ee0f202/roles/2eea147ab8b1426294a08106410f6d45` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:22.155543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e45b8-5d30-4122-9b0a-a035cc8922e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=985f95db5dac42d1808e299aca462b6b, user_id=a678fa6854274fcf84086a7a0ee0f202, role_id=2eea147ab8b1426294a08106410f6d45)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:22.156975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e45b8-5d30-4122-9b0a-a035cc8922e3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:22.172899 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 561/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:03:22 2022] PUT /identity/v3/projects/985f95db5dac42d1808e299aca462b6b/users/a678fa6854274fcf84086a7a0ee0f202/roles/2eea147ab8b1426294a08106410f6d45 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:03:24.116891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6503902-26c0-40fb-96a2-ce44370c8ac2 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:24.117857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6503902-26c0-40fb-96a2-ce44370c8ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:24.117857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6503902-26c0-40fb-96a2-ce44370c8ac2 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:24.119995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 563/1124] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:24.125323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd28ea-3bf5-4650-9fe2-29fc05a50269 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:24.125796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd28ea-3bf5-4650-9fe2-29fc05a50269 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:24.125925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd28ea-3bf5-4650-9fe2-29fc05a50269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:24.144813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-24cd28ea-3bf5-4650-9fe2-29fc05a50269 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:24.146010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-24cd28ea-3bf5-4650-9fe2-29fc05a50269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:24.210145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:24.215662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 562/1125] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:24.238940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6503902-26c0-40fb-96a2-ce44370c8ac2 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:24.349141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6503902-26c0-40fb-96a2-ce44370c8ac2 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:24.385344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd112ac-de10-4c27-ba1a-172901c5ed3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:24.386165 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd112ac-de10-4c27-ba1a-172901c5ed3e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:24.386455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd112ac-de10-4c27-ba1a-172901c5ed3e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:24.386720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd112ac-de10-4c27-ba1a-172901c5ed3e None admin] PATH_INFO: `/v3/roles/2eea147ab8b1426294a08106410f6d45` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:24.388279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd112ac-de10-4c27-ba1a-172901c5ed3e None admin] RBAC: Authorizing `identity:get_role(role_id=2eea147ab8b1426294a08106410f6d45)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:24.389628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd112ac-de10-4c27-ba1a-172901c5ed3e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:24.391012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 564/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:24 2022] GET /identity/v3/roles/2eea147ab8b1426294a08106410f6d45 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:24.397794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd28ea-3bf5-4650-9fe2-29fc05a50269 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:24.510917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd28ea-3bf5-4650-9fe2-29fc05a50269 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:24.555946 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859384e-24d0-49f1-8feb-bc4b8f7d0575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:24.557252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859384e-24d0-49f1-8feb-bc4b8f7d0575 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:24.557666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859384e-24d0-49f1-8feb-bc4b8f7d0575 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:24.558112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859384e-24d0-49f1-8feb-bc4b8f7d0575 None admin] PATH_INFO: `/v3/users/a678fa6854274fcf84086a7a0ee0f202` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:24.559878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859384e-24d0-49f1-8feb-bc4b8f7d0575 None admin] RBAC: Authorizing `identity:get_user(user_id=a678fa6854274fcf84086a7a0ee0f202)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:24.562154 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859384e-24d0-49f1-8feb-bc4b8f7d0575 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:24.564229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 563/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:24 2022] GET /identity/v3/users/a678fa6854274fcf84086a7a0ee0f202 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:24.571717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd112ac-de10-4c27-ba1a-172901c5ed3e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:24.611126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:24.691677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd112ac-de10-4c27-ba1a-172901c5ed3e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:24.727561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b230e-0d53-4070-b8b3-ae3a5f9b08f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:24.728195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b230e-0d53-4070-b8b3-ae3a5f9b08f2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:24.728479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b230e-0d53-4070-b8b3-ae3a5f9b08f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:24.728755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b230e-0d53-4070-b8b3-ae3a5f9b08f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:24.796963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b230e-0d53-4070-b8b3-ae3a5f9b08f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:24.798453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b230e-0d53-4070-b8b3-ae3a5f9b08f2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:24.832536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:24.912757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 565/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:24.918972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859384e-24d0-49f1-8feb-bc4b8f7d0575 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:24.992424 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:25.029199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859384e-24d0-49f1-8feb-bc4b8f7d0575 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:25.064002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255baa0-5d75-48db-8b6d-82335aace3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:25.064848 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255baa0-5d75-48db-8b6d-82335aace3ea None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:25.065123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255baa0-5d75-48db-8b6d-82335aace3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:25.065396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255baa0-5d75-48db-8b6d-82335aace3ea None admin] PATH_INFO: `/v3/projects/985f95db5dac42d1808e299aca462b6b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:25.066647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255baa0-5d75-48db-8b6d-82335aace3ea None admin] RBAC: Authorizing `identity:get_project(project_id=985f95db5dac42d1808e299aca462b6b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:25.068107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255baa0-5d75-48db-8b6d-82335aace3ea None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:25.069810 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 564/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:24 2022] GET /identity/v3/projects/985f95db5dac42d1808e299aca462b6b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:25.076876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b230e-0d53-4070-b8b3-ae3a5f9b08f2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:25.198624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b230e-0d53-4070-b8b3-ae3a5f9b08f2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:25.236396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05b8c6-257c-4adf-82f8-0f790d9a0762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:25.237040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05b8c6-257c-4adf-82f8-0f790d9a0762 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:25.237290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05b8c6-257c-4adf-82f8-0f790d9a0762 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:25.237618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05b8c6-257c-4adf-82f8-0f790d9a0762 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:25.238451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05b8c6-257c-4adf-82f8-0f790d9a0762 None admin] RBAC: Adding query filter params (role.id=2eea147ab8b1426294a08106410f6d45, scope.project.id=985f95db5dac42d1808e299aca462b6b, user.id=a678fa6854274fcf84086a7a0ee0f202) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:25.238735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05b8c6-257c-4adf-82f8-0f790d9a0762 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:25.240190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05b8c6-257c-4adf-82f8-0f790d9a0762 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:25.243151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:25.246515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 566/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:03:25 2022] GET /identity/v3/role_assignments?user.id=a678fa6854274fcf84086a7a0ee0f202&scope.project.id=985f95db5dac42d1808e299aca462b6b&role.id=2eea147ab8b1426294a08106410f6d45 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:27.267349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccbbbb-6485-4e69-b2bc-7f7e971876c5 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:27.268221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccbbbb-6485-4e69-b2bc-7f7e971876c5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:27.268221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccbbbb-6485-4e69-b2bc-7f7e971876c5 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:27.270794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 565/1131] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:27.277012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdabbd-5868-4eb8-b999-78711c0d1f78 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:27.277722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdabbd-5868-4eb8-b999-78711c0d1f78 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:27.277862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdabbd-5868-4eb8-b999-78711c0d1f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:27.303094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0fcdabbd-5868-4eb8-b999-78711c0d1f78 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:27.304265 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0fcdabbd-5868-4eb8-b999-78711c0d1f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:27.376255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 567/1132] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:27.400079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccbbbb-6485-4e69-b2bc-7f7e971876c5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:27.524520 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccbbbb-6485-4e69-b2bc-7f7e971876c5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:27.562294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92555ed9-5c9e-4c61-9457-3ec53024651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:27.563071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92555ed9-5c9e-4c61-9457-3ec53024651d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:27.563347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92555ed9-5c9e-4c61-9457-3ec53024651d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:27.563613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92555ed9-5c9e-4c61-9457-3ec53024651d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:27.564826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92555ed9-5c9e-4c61-9457-3ec53024651d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:27.566487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92555ed9-5c9e-4c61-9457-3ec53024651d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:27.568240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 566/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:27.574790 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdabbd-5868-4eb8-b999-78711c0d1f78 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:27.693531 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdabbd-5868-4eb8-b999-78711c0d1f78 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:27.722876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:27.729354 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a6994-bb5d-47f6-9e5f-621f6c25efbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:27.730044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6994-bb5d-47f6-9e5f-621f6c25efbf None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:27.730333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6994-bb5d-47f6-9e5f-621f6c25efbf None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:27.730609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6994-bb5d-47f6-9e5f-621f6c25efbf None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:27.731313 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a6994-bb5d-47f6-9e5f-621f6c25efbf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:27.732751 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a6994-bb5d-47f6-9e5f-621f6c25efbf None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:27.762333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 568/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:03:27 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:03:29.803738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd2275-ea47-45aa-8825-12030d01afeb None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:29.804644 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd2275-ea47-45aa-8825-12030d01afeb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:29.804644 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd2275-ea47-45aa-8825-12030d01afeb None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:29.806902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 567/1135] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:29.812567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d4a44-916e-4426-bb49-0de0f2c6e826 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:29.813022 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d4a44-916e-4426-bb49-0de0f2c6e826 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:29.813120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d4a44-916e-4426-bb49-0de0f2c6e826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:29.832192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-158d4a44-916e-4426-bb49-0de0f2c6e826 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:29.833355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-158d4a44-916e-4426-bb49-0de0f2c6e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:29.910283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 569/1136] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:29.947142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd2275-ea47-45aa-8825-12030d01afeb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:29.953337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:30.067896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd2275-ea47-45aa-8825-12030d01afeb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:30.116625 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3cebb-468b-4a6d-b856-cfa5d3f64719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:30.117422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3cebb-468b-4a6d-b856-cfa5d3f64719 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:30.117709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3cebb-468b-4a6d-b856-cfa5d3f64719 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:30.118024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3cebb-468b-4a6d-b856-cfa5d3f64719 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:30.119238 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3cebb-468b-4a6d-b856-cfa5d3f64719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:30.120716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3cebb-468b-4a6d-b856-cfa5d3f64719 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:30.122373 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 568/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:03:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:30.129051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d4a44-916e-4426-bb49-0de0f2c6e826 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:30.233790 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d4a44-916e-4426-bb49-0de0f2c6e826 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:30.247534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:30.270869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176cd9d-7c2e-4908-831c-901f10ce1572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:30.271545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176cd9d-7c2e-4908-831c-901f10ce1572 None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:30.271835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176cd9d-7c2e-4908-831c-901f10ce1572 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:30.272119 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176cd9d-7c2e-4908-831c-901f10ce1572 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:30.272826 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176cd9d-7c2e-4908-831c-901f10ce1572 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:30.274321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176cd9d-7c2e-4908-831c-901f10ce1572 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:30.295458 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 570/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:03:30 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:03:32.306669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b38ce-cf33-4b85-8654-fb0adb3c5961 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:32.307264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b38ce-cf33-4b85-8654-fb0adb3c5961 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:32.307264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b38ce-cf33-4b85-8654-fb0adb3c5961 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:32.308596 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 569/1139] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:32.314888 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a10cc4-5f67-4ac6-9185-097a760faca5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:32.315344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a10cc4-5f67-4ac6-9185-097a760faca5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:32.315464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a10cc4-5f67-4ac6-9185-097a760faca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:32.339567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-45a10cc4-5f67-4ac6-9185-097a760faca5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:32.340755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-45a10cc4-5f67-4ac6-9185-097a760faca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:32.412209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 571/1140] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:32.436545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b38ce-cf33-4b85-8654-fb0adb3c5961 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:32.520402 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:32.553717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b38ce-cf33-4b85-8654-fb0adb3c5961 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:32.588674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb9f2e-c9de-4b70-a520-5bc4ab325d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:32.589452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb9f2e-c9de-4b70-a520-5bc4ab325d99 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:32.589737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb9f2e-c9de-4b70-a520-5bc4ab325d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:32.590041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb9f2e-c9de-4b70-a520-5bc4ab325d99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:32.601122 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb9f2e-c9de-4b70-a520-5bc4ab325d99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e0eb9f2e-c9de-4b70-a520-5bc4ab325d99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:32.608559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:32.609697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:32.610784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 570/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:03:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:32.616490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a10cc4-5f67-4ac6-9185-097a760faca5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:32.734244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:32.738801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a10cc4-5f67-4ac6-9185-097a760faca5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:32.778829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cea28e-117f-4caa-9997-d7ac33c30a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:32.779487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea28e-117f-4caa-9997-d7ac33c30a9a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:32.779767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea28e-117f-4caa-9997-d7ac33c30a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:32.780038 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea28e-117f-4caa-9997-d7ac33c30a9a None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:32.780783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cea28e-117f-4caa-9997-d7ac33c30a9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:32.781089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cea28e-117f-4caa-9997-d7ac33c30a9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:32.782506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cea28e-117f-4caa-9997-d7ac33c30a9a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:32.790818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 572/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:03:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:32.799209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb9f2e-c9de-4b70-a520-5bc4ab325d99 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:32.906726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb9f2e-c9de-4b70-a520-5bc4ab325d99 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:32.936122 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:32.941876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673c465-4b83-416e-b0a3-99f54737f7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:32.942773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673c465-4b83-416e-b0a3-99f54737f7bc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:32.943052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673c465-4b83-416e-b0a3-99f54737f7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:32.943323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673c465-4b83-416e-b0a3-99f54737f7bc None admin] PATH_INFO: `/v3/users/3565467f0d1e4a75a3c0563dcdc1033b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:32.957160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673c465-4b83-416e-b0a3-99f54737f7bc None admin] RBAC: Authorizing `identity:get_user(user_id=3565467f0d1e4a75a3c0563dcdc1033b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:32.958636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673c465-4b83-416e-b0a3-99f54737f7bc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:32.960276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 571/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:32 2022] GET /identity/v3/users/3565467f0d1e4a75a3c0563dcdc1033b => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:32.966314 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cea28e-117f-4caa-9997-d7ac33c30a9a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:33.081814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cea28e-117f-4caa-9997-d7ac33c30a9a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:33.122160 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949938b7-a197-455c-95b2-e602fbaafd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:33.122805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949938b7-a197-455c-95b2-e602fbaafd86 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:33.123091 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949938b7-a197-455c-95b2-e602fbaafd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:33.123369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949938b7-a197-455c-95b2-e602fbaafd86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:33.154630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:33.195624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949938b7-a197-455c-95b2-e602fbaafd86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:33.197066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949938b7-a197-455c-95b2-e602fbaafd86 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:33.310458 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 573/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:33.321321 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673c465-4b83-416e-b0a3-99f54737f7bc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:33.440000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673c465-4b83-416e-b0a3-99f54737f7bc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:33.477011 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca5327-e55b-4d00-9b52-b551e9c468eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:33.477868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca5327-e55b-4d00-9b52-b551e9c468eb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:33.478170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca5327-e55b-4d00-9b52-b551e9c468eb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:33.478446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca5327-e55b-4d00-9b52-b551e9c468eb None admin] PATH_INFO: `/v3/projects/f1df233dfe2b4d84b02f1116b5da4f2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:33.479726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca5327-e55b-4d00-9b52-b551e9c468eb None admin] RBAC: Authorizing `identity:get_project(project_id=f1df233dfe2b4d84b02f1116b5da4f2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:33.481156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca5327-e55b-4d00-9b52-b551e9c468eb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:33.482923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 572/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:33 2022] GET /identity/v3/projects/f1df233dfe2b4d84b02f1116b5da4f2f => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:33.493082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949938b7-a197-455c-95b2-e602fbaafd86 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:33.563549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:33.605236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949938b7-a197-455c-95b2-e602fbaafd86 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:33.640951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b162db-0180-4881-9128-f3da38ea48bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:33.641591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b162db-0180-4881-9128-f3da38ea48bc None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:33.641872 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b162db-0180-4881-9128-f3da38ea48bc None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:33.642178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b162db-0180-4881-9128-f3da38ea48bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:33.643019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b162db-0180-4881-9128-f3da38ea48bc None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=f1df233dfe2b4d84b02f1116b5da4f2f, user.id=3565467f0d1e4a75a3c0563dcdc1033b) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:33.643324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b162db-0180-4881-9128-f3da38ea48bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:33.644783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b162db-0180-4881-9128-f3da38ea48bc None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:33.650346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 574/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:03:33 2022] GET /identity/v3/role_assignments?user.id=3565467f0d1e4a75a3c0563dcdc1033b&scope.project.id=f1df233dfe2b4d84b02f1116b5da4f2f&role.id=2302517080ac42578aa81c606dd0ff2f => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:35.645639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679926b-e883-418a-903c-d54dd68aaff6 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:35.646714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679926b-e883-418a-903c-d54dd68aaff6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:35.646714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679926b-e883-418a-903c-d54dd68aaff6 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:35.648781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 573/1147] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:35.655129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5821a-a963-46f6-83bc-4ae694bc36b2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:35.655581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5821a-a963-46f6-83bc-4ae694bc36b2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:35.655802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5821a-a963-46f6-83bc-4ae694bc36b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:35.674010 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-95c5821a-a963-46f6-83bc-4ae694bc36b2 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:35.675191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-95c5821a-a963-46f6-83bc-4ae694bc36b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:35.744089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 575/1148] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:35.766940 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679926b-e883-418a-903c-d54dd68aaff6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:35.830173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:35.875750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679926b-e883-418a-903c-d54dd68aaff6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:35.914310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2312a00-404b-4e2f-9674-b03112900389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:35.915035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2312a00-404b-4e2f-9674-b03112900389 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:35.915322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2312a00-404b-4e2f-9674-b03112900389 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:35.915600 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2312a00-404b-4e2f-9674-b03112900389 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:35.926174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2312a00-404b-4e2f-9674-b03112900389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b2312a00-404b-4e2f-9674-b03112900389 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:35.933728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:35.934813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:35.935994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 574/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:03:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:35.941294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5821a-a963-46f6-83bc-4ae694bc36b2 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:36.000212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:36.046933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5821a-a963-46f6-83bc-4ae694bc36b2 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:36.079907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f30e7-cd6b-41a4-ba09-74a2ca2c9f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:36.080562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f30e7-cd6b-41a4-ba09-74a2ca2c9f16 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:36.080841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f30e7-cd6b-41a4-ba09-74a2ca2c9f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:36.081118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f30e7-cd6b-41a4-ba09-74a2ca2c9f16 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:36.081923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f30e7-cd6b-41a4-ba09-74a2ca2c9f16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:36.082205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f30e7-cd6b-41a4-ba09-74a2ca2c9f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:36.083599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f30e7-cd6b-41a4-ba09-74a2ca2c9f16 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:36.091443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 576/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:03:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:36.097342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2312a00-404b-4e2f-9674-b03112900389 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:36.216389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2312a00-404b-4e2f-9674-b03112900389 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:36.235686 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:36.264332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87885d-bce0-4649-a3cb-60613fb650f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:36.265580 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87885d-bce0-4649-a3cb-60613fb650f7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:36.265952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87885d-bce0-4649-a3cb-60613fb650f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:36.266336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87885d-bce0-4649-a3cb-60613fb650f7 None admin] PATH_INFO: `/v3/users/3565467f0d1e4a75a3c0563dcdc1033b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:36.268238 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87885d-bce0-4649-a3cb-60613fb650f7 None admin] RBAC: Authorizing `identity:get_user(user_id=3565467f0d1e4a75a3c0563dcdc1033b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:36.270115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87885d-bce0-4649-a3cb-60613fb650f7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:36.271728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 575/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:36 2022] GET /identity/v3/users/3565467f0d1e4a75a3c0563dcdc1033b => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:36.278006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f30e7-cd6b-41a4-ba09-74a2ca2c9f16 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:36.382470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f30e7-cd6b-41a4-ba09-74a2ca2c9f16 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:36.398685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:36.417497 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9d04a-eb58-44ca-af96-0235b0c90888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:36.418381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d04a-eb58-44ca-af96-0235b0c90888 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:36.418670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d04a-eb58-44ca-af96-0235b0c90888 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:36.418946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9d04a-eb58-44ca-af96-0235b0c90888 None admin] PATH_INFO: `/v3/projects/f1df233dfe2b4d84b02f1116b5da4f2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:36.420134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9d04a-eb58-44ca-af96-0235b0c90888 None admin] RBAC: Authorizing `identity:get_project(project_id=f1df233dfe2b4d84b02f1116b5da4f2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:36.421568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9d04a-eb58-44ca-af96-0235b0c90888 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:36.423251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 577/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:36 2022] GET /identity/v3/projects/f1df233dfe2b4d84b02f1116b5da4f2f => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:36.432937 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87885d-bce0-4649-a3cb-60613fb650f7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:36.534381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87885d-bce0-4649-a3cb-60613fb650f7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:36.569553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da93fa-c51f-4140-9295-e5c646790f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:36.570455 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da93fa-c51f-4140-9295-e5c646790f35 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:36.570732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da93fa-c51f-4140-9295-e5c646790f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:36.571006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da93fa-c51f-4140-9295-e5c646790f35 None admin] PATH_INFO: `/v3/projects/f1df233dfe2b4d84b02f1116b5da4f2f/users/3565467f0d1e4a75a3c0563dcdc1033b/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:36.573181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da93fa-c51f-4140-9295-e5c646790f35 None admin] RBAC: Authorizing `identity:create_grant(project_id=f1df233dfe2b4d84b02f1116b5da4f2f, user_id=3565467f0d1e4a75a3c0563dcdc1033b, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:36.574701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da93fa-c51f-4140-9295-e5c646790f35 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:36.590925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 576/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:03:36 2022] PUT /identity/v3/projects/f1df233dfe2b4d84b02f1116b5da4f2f/users/3565467f0d1e4a75a3c0563dcdc1033b/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:03:38.572229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069833e-6097-4fe6-ab48-8f975ffb7ed8 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:38.573087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069833e-6097-4fe6-ab48-8f975ffb7ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:38.573087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069833e-6097-4fe6-ab48-8f975ffb7ed8 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:38.575235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 578/1154] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:38.581474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c292ad-3609-4757-a450-0aedacfeb3c9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:38.582129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c292ad-3609-4757-a450-0aedacfeb3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:38.582306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c292ad-3609-4757-a450-0aedacfeb3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:38.600404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-55c292ad-3609-4757-a450-0aedacfeb3c9 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:38.601593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-55c292ad-3609-4757-a450-0aedacfeb3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:38.646094 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:38.671915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 577/1155] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:38.695438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069833e-6097-4fe6-ab48-8f975ffb7ed8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:38.808713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069833e-6097-4fe6-ab48-8f975ffb7ed8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:38.844357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7d746-cafd-4f2a-8d1a-06f164047b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:38.845149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7d746-cafd-4f2a-8d1a-06f164047b43 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:38.845431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7d746-cafd-4f2a-8d1a-06f164047b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:38.845712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7d746-cafd-4f2a-8d1a-06f164047b43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:38.856321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7d746-cafd-4f2a-8d1a-06f164047b43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-a2f7d746-cafd-4f2a-8d1a-06f164047b43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:38.863933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:38.865034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:38.866084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 579/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:03:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:38.873593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c292ad-3609-4757-a450-0aedacfeb3c9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:38.975371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c292ad-3609-4757-a450-0aedacfeb3c9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:39.010658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c80ee-6405-4f53-bc7c-026dbefba6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:39.011315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c80ee-6405-4f53-bc7c-026dbefba6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:39.011594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c80ee-6405-4f53-bc7c-026dbefba6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:39.011866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c80ee-6405-4f53-bc7c-026dbefba6b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:39.012613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c80ee-6405-4f53-bc7c-026dbefba6b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:39.012922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c80ee-6405-4f53-bc7c-026dbefba6b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:39.014342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c80ee-6405-4f53-bc7c-026dbefba6b6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:39.022976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 578/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:03:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:39.032422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7d746-cafd-4f2a-8d1a-06f164047b43 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:39.088738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:39.147374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7d746-cafd-4f2a-8d1a-06f164047b43 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:39.181763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb85c67-d558-46d8-8921-c30ea7f219d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:39.182659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb85c67-d558-46d8-8921-c30ea7f219d4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:39.182948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb85c67-d558-46d8-8921-c30ea7f219d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:39.183214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb85c67-d558-46d8-8921-c30ea7f219d4 None admin] PATH_INFO: `/v3/users/3565467f0d1e4a75a3c0563dcdc1033b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:39.184422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb85c67-d558-46d8-8921-c30ea7f219d4 None admin] RBAC: Authorizing `identity:get_user(user_id=3565467f0d1e4a75a3c0563dcdc1033b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:39.185856 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb85c67-d558-46d8-8921-c30ea7f219d4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:39.187498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 580/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:39 2022] GET /identity/v3/users/3565467f0d1e4a75a3c0563dcdc1033b => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:39.193730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c80ee-6405-4f53-bc7c-026dbefba6b6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:39.234526 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:39.299577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c80ee-6405-4f53-bc7c-026dbefba6b6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:39.342434 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a0f77-97dd-4919-990a-89f338ac3943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:39.343077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a0f77-97dd-4919-990a-89f338ac3943 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:39.343361 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a0f77-97dd-4919-990a-89f338ac3943 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:39.343634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a0f77-97dd-4919-990a-89f338ac3943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:39.420182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a0f77-97dd-4919-990a-89f338ac3943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:39.421559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a0f77-97dd-4919-990a-89f338ac3943 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:39.459972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:39.523841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 579/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:39.536189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb85c67-d558-46d8-8921-c30ea7f219d4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:39.643238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb85c67-d558-46d8-8921-c30ea7f219d4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:39.656187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:39.680945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d98afe-c0be-430c-a09b-461784e6da54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:39.681733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98afe-c0be-430c-a09b-461784e6da54 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:39.682056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98afe-c0be-430c-a09b-461784e6da54 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:39.682337 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d98afe-c0be-430c-a09b-461784e6da54 None admin] PATH_INFO: `/v3/projects/f1df233dfe2b4d84b02f1116b5da4f2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:39.683573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d98afe-c0be-430c-a09b-461784e6da54 None admin] RBAC: Authorizing `identity:get_project(project_id=f1df233dfe2b4d84b02f1116b5da4f2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:39.685015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d98afe-c0be-430c-a09b-461784e6da54 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:39.686739 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 581/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:39 2022] GET /identity/v3/projects/f1df233dfe2b4d84b02f1116b5da4f2f => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:39.693657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a0f77-97dd-4919-990a-89f338ac3943 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:39.802375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a0f77-97dd-4919-990a-89f338ac3943 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:39.837412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10759aef-9df8-4443-a557-e4aff7837999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:39.838092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10759aef-9df8-4443-a557-e4aff7837999 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:39.838377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10759aef-9df8-4443-a557-e4aff7837999 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:39.838654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10759aef-9df8-4443-a557-e4aff7837999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:39.839491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10759aef-9df8-4443-a557-e4aff7837999 None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=f1df233dfe2b4d84b02f1116b5da4f2f, user.id=3565467f0d1e4a75a3c0563dcdc1033b) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:39.839808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10759aef-9df8-4443-a557-e4aff7837999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:39.841240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10759aef-9df8-4443-a557-e4aff7837999 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:39.847325 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 580/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:03:39 2022] GET /identity/v3/role_assignments?user.id=3565467f0d1e4a75a3c0563dcdc1033b&scope.project.id=f1df233dfe2b4d84b02f1116b5da4f2f&role.id=2302517080ac42578aa81c606dd0ff2f => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:41.851853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07786114-02cc-42ff-85e6-dc2815020a8b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:41.852411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07786114-02cc-42ff-85e6-dc2815020a8b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:41.852411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07786114-02cc-42ff-85e6-dc2815020a8b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:41.854576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 582/1162] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:41.861202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b87f2c-c5cf-43ea-893d-f8e221ae5a0b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:41.861639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b87f2c-c5cf-43ea-893d-f8e221ae5a0b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:41.861777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b87f2c-c5cf-43ea-893d-f8e221ae5a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:41.870707 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:41.887817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-94b87f2c-c5cf-43ea-893d-f8e221ae5a0b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:41.889018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-94b87f2c-c5cf-43ea-893d-f8e221ae5a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:41.962957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 581/1163] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:41.986788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07786114-02cc-42ff-85e6-dc2815020a8b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:42.102973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07786114-02cc-42ff-85e6-dc2815020a8b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:42.141859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbe12e-de4f-42d4-bfe9-44a3a65383b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:42.142524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbe12e-de4f-42d4-bfe9-44a3a65383b8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:42.142811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbe12e-de4f-42d4-bfe9-44a3a65383b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:42.143087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbe12e-de4f-42d4-bfe9-44a3a65383b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:42.192394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:42.214218 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbe12e-de4f-42d4-bfe9-44a3a65383b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:42.215706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbe12e-de4f-42d4-bfe9-44a3a65383b8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:42.316436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 583/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:42.322687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b87f2c-c5cf-43ea-893d-f8e221ae5a0b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:42.436858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b87f2c-c5cf-43ea-893d-f8e221ae5a0b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:42.470235 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:42.477112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6376e6-7609-4c77-80ec-ad509c836b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:42.477915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6376e6-7609-4c77-80ec-ad509c836b1e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:42.478223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6376e6-7609-4c77-80ec-ad509c836b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:42.478495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6376e6-7609-4c77-80ec-ad509c836b1e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:42.485398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6376e6-7609-4c77-80ec-ad509c836b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:42.486880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6376e6-7609-4c77-80ec-ad509c836b1e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-6c6376e6-7609-4c77-80ec-ad509c836b1e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:42.501135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:42.502328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:42.503354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 582/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 4 09:03:42 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:42.509480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbe12e-de4f-42d4-bfe9-44a3a65383b8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:42.614134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:42.640255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbe12e-de4f-42d4-bfe9-44a3a65383b8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:42.676098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5eeef-6f38-46d5-bb84-1ef240c07762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:42.677050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5eeef-6f38-46d5-bb84-1ef240c07762 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:42.677455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5eeef-6f38-46d5-bb84-1ef240c07762 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:42.677819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5eeef-6f38-46d5-bb84-1ef240c07762 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:42.678963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5eeef-6f38-46d5-bb84-1ef240c07762 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:42.679404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5eeef-6f38-46d5-bb84-1ef240c07762 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:42.681531 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5eeef-6f38-46d5-bb84-1ef240c07762 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:42.693030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 584/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 4 09:03:42 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:44.758966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe743e-8e66-4830-84b0-04c5ef543d0d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:44.759875 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe743e-8e66-4830-84b0-04c5ef543d0d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:44.759875 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe743e-8e66-4830-84b0-04c5ef543d0d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:44.762151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 583/1167] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:44.768299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6da61-1f00-4e78-84f6-ee3bad97ceb9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:44.768776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6da61-1f00-4e78-84f6-ee3bad97ceb9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:44.768871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6da61-1f00-4e78-84f6-ee3bad97ceb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:44.793049 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8cf6da61-1f00-4e78-84f6-ee3bad97ceb9 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:44.794271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8cf6da61-1f00-4e78-84f6-ee3bad97ceb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:44.865304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 585/1168] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:44.888660 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe743e-8e66-4830-84b0-04c5ef543d0d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:44.999348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe743e-8e66-4830-84b0-04c5ef543d0d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:45.035036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992e9ba-d4e2-40c0-a383-de8bf1a65b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:45.035840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992e9ba-d4e2-40c0-a383-de8bf1a65b91 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:45.036120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992e9ba-d4e2-40c0-a383-de8bf1a65b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:45.036407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992e9ba-d4e2-40c0-a383-de8bf1a65b91 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:45.037154 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992e9ba-d4e2-40c0-a383-de8bf1a65b91 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:45.038565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992e9ba-d4e2-40c0-a383-de8bf1a65b91 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:45.058247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 584/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Mon Jul 4 09:03:44 2022] POST /identity/v3/domains => generated 267 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:03:47.076767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e14e7-6bae-4c99-82cb-a35a53b2867e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:47.077343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e14e7-6bae-4c99-82cb-a35a53b2867e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:47.077343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e14e7-6bae-4c99-82cb-a35a53b2867e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:47.078721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 586/1170] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:47.085087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f0f0d-9b78-41ae-8cd1-d599a3d11894 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:47.085914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f0f0d-9b78-41ae-8cd1-d599a3d11894 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:47.086012 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f0f0d-9b78-41ae-8cd1-d599a3d11894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:47.108813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0b8f0f0d-9b78-41ae-8cd1-d599a3d11894 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:47.110162 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0b8f0f0d-9b78-41ae-8cd1-d599a3d11894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:47.165521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:47.186991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 585/1171] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:47.210701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e14e7-6bae-4c99-82cb-a35a53b2867e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:47.293909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:47.325141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e14e7-6bae-4c99-82cb-a35a53b2867e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:47.365505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d2b47-da50-4b67-85d9-24077a1c6ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:47.366383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d2b47-da50-4b67-85d9-24077a1c6ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:47.366665 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d2b47-da50-4b67-85d9-24077a1c6ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:47.366938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d2b47-da50-4b67-85d9-24077a1c6ab5 None admin] PATH_INFO: `/v3/domains/6543ecf0d6864eaa97dfcd7e4f3f623c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:47.378599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d2b47-da50-4b67-85d9-24077a1c6ab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6543ecf0d6864eaa97dfcd7e4f3f623c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:47.380480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d2b47-da50-4b67-85d9-24077a1c6ab5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:47.382322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 587/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Jul 4 09:03:47 2022] GET /identity/v3/domains/6543ecf0d6864eaa97dfcd7e4f3f623c => generated 267 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:47.388560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f0f0d-9b78-41ae-8cd1-d599a3d11894 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:47.501219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f0f0d-9b78-41ae-8cd1-d599a3d11894 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:47.537928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fe9ba-c23a-4b6c-81e2-cc9940440f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:47.538585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fe9ba-c23a-4b6c-81e2-cc9940440f0d None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:47.538868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fe9ba-c23a-4b6c-81e2-cc9940440f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:47.539145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fe9ba-c23a-4b6c-81e2-cc9940440f0d None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:47.539858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fe9ba-c23a-4b6c-81e2-cc9940440f0d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:47.541295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fe9ba-c23a-4b6c-81e2-cc9940440f0d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:47.570378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 586/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:03:47 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:03:49.585207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6168d-a075-4047-8e43-79f998533657 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:49.585783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6168d-a075-4047-8e43-79f998533657 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:49.585783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6168d-a075-4047-8e43-79f998533657 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:49.587866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 588/1174] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:49.593947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830ef7c-6e63-4b40-bb64-b9aad971c0b2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:49.594366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830ef7c-6e63-4b40-bb64-b9aad971c0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:49.594516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830ef7c-6e63-4b40-bb64-b9aad971c0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:49.616692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9830ef7c-6e63-4b40-bb64-b9aad971c0b2 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:49.617923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9830ef7c-6e63-4b40-bb64-b9aad971c0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:49.648185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:49.700297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 587/1175] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:49.725475 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6168d-a075-4047-8e43-79f998533657 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:49.837774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6168d-a075-4047-8e43-79f998533657 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:49.873071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9b0c3-1ebf-43cc-8b62-c050f28adf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:49.873989 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9b0c3-1ebf-43cc-8b62-c050f28adf4a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:49.874234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9b0c3-1ebf-43cc-8b62-c050f28adf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:49.874513 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9b0c3-1ebf-43cc-8b62-c050f28adf4a None admin] PATH_INFO: `/v3/domains/6543ecf0d6864eaa97dfcd7e4f3f623c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:49.876485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9b0c3-1ebf-43cc-8b62-c050f28adf4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=6543ecf0d6864eaa97dfcd7e4f3f623c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:49.879379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9b0c3-1ebf-43cc-8b62-c050f28adf4a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:49.881021 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 589/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Jul 4 09:03:49 2022] GET /identity/v3/domains/6543ecf0d6864eaa97dfcd7e4f3f623c => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:49.887220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830ef7c-6e63-4b40-bb64-b9aad971c0b2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:49.994122 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830ef7c-6e63-4b40-bb64-b9aad971c0b2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:50.030356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78ab68-a840-4d1c-a561-c147125cbb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:50.031007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78ab68-a840-4d1c-a561-c147125cbb33 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:50.031283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78ab68-a840-4d1c-a561-c147125cbb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:50.031557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78ab68-a840-4d1c-a561-c147125cbb33 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:50.032257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78ab68-a840-4d1c-a561-c147125cbb33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:50.033681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78ab68-a840-4d1c-a561-c147125cbb33 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:50.056274 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 588/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:03:49 2022] POST /identity/v3/users => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:03:52.099582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921125c-ce70-467b-82ed-0c5d3ea921e7 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:52.100154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921125c-ce70-467b-82ed-0c5d3ea921e7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:52.100154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921125c-ce70-467b-82ed-0c5d3ea921e7 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:52.101520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 590/1178] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:03:52.107794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77d185-fad6-411f-a454-bdf723fd9aad None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:52.108508 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77d185-fad6-411f-a454-bdf723fd9aad None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:52.108650 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77d185-fad6-411f-a454-bdf723fd9aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:52.128912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ee77d185-fad6-411f-a454-bdf723fd9aad None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:52.130661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ee77d185-fad6-411f-a454-bdf723fd9aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:52.145679 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:52.212631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 589/1179] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:52.236648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921125c-ce70-467b-82ed-0c5d3ea921e7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:52.239670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:52.346200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921125c-ce70-467b-82ed-0c5d3ea921e7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:52.381538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b6079-38ae-4c6c-b843-fddca1223cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:52.382288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b6079-38ae-4c6c-b843-fddca1223cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:52.382568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b6079-38ae-4c6c-b843-fddca1223cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:52.382838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b6079-38ae-4c6c-b843-fddca1223cf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:52.401214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b6079-38ae-4c6c-b843-fddca1223cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ad4b6079-38ae-4c6c-b843-fddca1223cf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:52.412492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:52.413600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:52.414642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:52.418056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 591/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:03:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:52.419745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77d185-fad6-411f-a454-bdf723fd9aad None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:52.522271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77d185-fad6-411f-a454-bdf723fd9aad None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:52.559813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d82aa-7f00-45d1-b44f-4e07b06166b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:52.560467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d82aa-7f00-45d1-b44f-4e07b06166b0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:52.560757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d82aa-7f00-45d1-b44f-4e07b06166b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:52.561033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d82aa-7f00-45d1-b44f-4e07b06166b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:52.561788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d82aa-7f00-45d1-b44f-4e07b06166b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:52.562138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d82aa-7f00-45d1-b44f-4e07b06166b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:52.563517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d82aa-7f00-45d1-b44f-4e07b06166b0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:52.571844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 590/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:03:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:52.577747 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b6079-38ae-4c6c-b843-fddca1223cf0 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:52.643124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:52.694912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b6079-38ae-4c6c-b843-fddca1223cf0 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:52.730770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3242971-d7a0-454e-8e88-36431531f3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:52.731653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3242971-d7a0-454e-8e88-36431531f3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:52.731919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3242971-d7a0-454e-8e88-36431531f3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:52.732195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3242971-d7a0-454e-8e88-36431531f3a2 None admin] PATH_INFO: `/v3/users/7d0a16674f76445fb81eab0f76db806b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:52.746801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3242971-d7a0-454e-8e88-36431531f3a2 None admin] RBAC: Authorizing `identity:get_user(user_id=7d0a16674f76445fb81eab0f76db806b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:52.748247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3242971-d7a0-454e-8e88-36431531f3a2 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:52.749903 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 592/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:52 2022] GET /identity/v3/users/7d0a16674f76445fb81eab0f76db806b => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:52.756300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d82aa-7f00-45d1-b44f-4e07b06166b0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:52.772396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:52.871690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d82aa-7f00-45d1-b44f-4e07b06166b0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:52.912357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cd963-5cd9-4ab3-bc5a-ebcc0c1283d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:52.912926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cd963-5cd9-4ab3-bc5a-ebcc0c1283d1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:52.913211 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cd963-5cd9-4ab3-bc5a-ebcc0c1283d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:52.913484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cd963-5cd9-4ab3-bc5a-ebcc0c1283d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:52.982329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cd963-5cd9-4ab3-bc5a-ebcc0c1283d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:52.983770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cd963-5cd9-4ab3-bc5a-ebcc0c1283d1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:52.999387 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:53.086601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 591/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:53.095380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3242971-d7a0-454e-8e88-36431531f3a2 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:53.203530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3242971-d7a0-454e-8e88-36431531f3a2 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:53.224099 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:53.240114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5f797-dfd5-4130-8cb5-a6027912815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:53.240969 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f797-dfd5-4130-8cb5-a6027912815b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:53.241251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f797-dfd5-4130-8cb5-a6027912815b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:53.241527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f797-dfd5-4130-8cb5-a6027912815b None admin] PATH_INFO: `/v3/projects/3dfa40d3bc50431ba062dfe819ca58ca` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:53.242862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5f797-dfd5-4130-8cb5-a6027912815b None admin] RBAC: Authorizing `identity:get_project(project_id=3dfa40d3bc50431ba062dfe819ca58ca)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:53.244304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5f797-dfd5-4130-8cb5-a6027912815b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:53.246040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 593/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:53 2022] GET /identity/v3/projects/3dfa40d3bc50431ba062dfe819ca58ca => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:53.253052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cd963-5cd9-4ab3-bc5a-ebcc0c1283d1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:53.354498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cd963-5cd9-4ab3-bc5a-ebcc0c1283d1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:53.367638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:53.391103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab053e-dd4f-4c7d-b1d3-0b8adf505df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:53.391745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab053e-dd4f-4c7d-b1d3-0b8adf505df1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:53.392022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab053e-dd4f-4c7d-b1d3-0b8adf505df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:53.392304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab053e-dd4f-4c7d-b1d3-0b8adf505df1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:53.393133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab053e-dd4f-4c7d-b1d3-0b8adf505df1 None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=3dfa40d3bc50431ba062dfe819ca58ca, user.id=7d0a16674f76445fb81eab0f76db806b) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:53.393444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab053e-dd4f-4c7d-b1d3-0b8adf505df1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:53.395209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab053e-dd4f-4c7d-b1d3-0b8adf505df1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:53.400781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 592/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:03:53 2022] GET /identity/v3/role_assignments?user.id=7d0a16674f76445fb81eab0f76db806b&scope.project.id=3dfa40d3bc50431ba062dfe819ca58ca&role.id=2302517080ac42578aa81c606dd0ff2f => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:55.378907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931725-b2dd-495e-8690-cfa7692d9ecc None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:55.379811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931725-b2dd-495e-8690-cfa7692d9ecc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:55.379811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931725-b2dd-495e-8690-cfa7692d9ecc None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:55.381146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 594/1186] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:55.387614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be432416-1e8d-48ea-89a5-a0008d9d4a60 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:55.388343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be432416-1e8d-48ea-89a5-a0008d9d4a60 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:55.388859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be432416-1e8d-48ea-89a5-a0008d9d4a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:55.414255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-be432416-1e8d-48ea-89a5-a0008d9d4a60 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:55.415382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-be432416-1e8d-48ea-89a5-a0008d9d4a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:55.486423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 593/1187] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:55.516508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e931725-b2dd-495e-8690-cfa7692d9ecc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:55.630294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e931725-b2dd-495e-8690-cfa7692d9ecc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:55.667864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8605cc-ce51-472e-8dc6-3820d1dd8198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:55.668578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8605cc-ce51-472e-8dc6-3820d1dd8198 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:55.668866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8605cc-ce51-472e-8dc6-3820d1dd8198 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:55.669146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8605cc-ce51-472e-8dc6-3820d1dd8198 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:55.680609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8605cc-ce51-472e-8dc6-3820d1dd8198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-2c8605cc-ce51-472e-8dc6-3820d1dd8198 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:55.688118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:55.689236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:03:55.690185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 595/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:03:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:55.698532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be432416-1e8d-48ea-89a5-a0008d9d4a60 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:55.810245 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be432416-1e8d-48ea-89a5-a0008d9d4a60 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:55.818420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:55.848784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e65fb-4a01-49c9-bf26-e9e3ddc5431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:55.849473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e65fb-4a01-49c9-bf26-e9e3ddc5431e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:55.849756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e65fb-4a01-49c9-bf26-e9e3ddc5431e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:55.850066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e65fb-4a01-49c9-bf26-e9e3ddc5431e None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:55.850830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e65fb-4a01-49c9-bf26-e9e3ddc5431e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:55.851135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e65fb-4a01-49c9-bf26-e9e3ddc5431e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:55.852713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e65fb-4a01-49c9-bf26-e9e3ddc5431e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:55.862407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 594/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:03:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:55.868809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8605cc-ce51-472e-8dc6-3820d1dd8198 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:55.927587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:55.980164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8605cc-ce51-472e-8dc6-3820d1dd8198 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:56.015390 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ca0a7-9751-49f4-a0b5-45a8f3a3f639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:56.016249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca0a7-9751-49f4-a0b5-45a8f3a3f639 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:56.016526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca0a7-9751-49f4-a0b5-45a8f3a3f639 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:56.016798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca0a7-9751-49f4-a0b5-45a8f3a3f639 None admin] PATH_INFO: `/v3/users/7d0a16674f76445fb81eab0f76db806b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:56.018127 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ca0a7-9751-49f4-a0b5-45a8f3a3f639 None admin] RBAC: Authorizing `identity:get_user(user_id=7d0a16674f76445fb81eab0f76db806b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:56.019578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ca0a7-9751-49f4-a0b5-45a8f3a3f639 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:56.021174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 596/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:55 2022] GET /identity/v3/users/7d0a16674f76445fb81eab0f76db806b => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:56.029913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e65fb-4a01-49c9-bf26-e9e3ddc5431e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:56.145623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e65fb-4a01-49c9-bf26-e9e3ddc5431e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:56.185211 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ad0f0-1c12-411d-9675-3f6dd81676d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:56.186042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ad0f0-1c12-411d-9675-3f6dd81676d4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:56.186325 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ad0f0-1c12-411d-9675-3f6dd81676d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:56.186607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ad0f0-1c12-411d-9675-3f6dd81676d4 None admin] PATH_INFO: `/v3/projects/3dfa40d3bc50431ba062dfe819ca58ca` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:56.187846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ad0f0-1c12-411d-9675-3f6dd81676d4 None admin] RBAC: Authorizing `identity:get_project(project_id=3dfa40d3bc50431ba062dfe819ca58ca)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:56.189290 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ad0f0-1c12-411d-9675-3f6dd81676d4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:56.191051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 595/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:56 2022] GET /identity/v3/projects/3dfa40d3bc50431ba062dfe819ca58ca => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:03:56.197821 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ca0a7-9751-49f4-a0b5-45a8f3a3f639 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:56.314651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ca0a7-9751-49f4-a0b5-45a8f3a3f639 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:56.330972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:56.350427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60374f60-1baf-4deb-bfcf-59a1fff4a6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:56.351305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60374f60-1baf-4deb-bfcf-59a1fff4a6a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:56.351586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60374f60-1baf-4deb-bfcf-59a1fff4a6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:56.351861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60374f60-1baf-4deb-bfcf-59a1fff4a6a1 None admin] PATH_INFO: `/v3/projects/3dfa40d3bc50431ba062dfe819ca58ca/users/7d0a16674f76445fb81eab0f76db806b/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:56.353933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60374f60-1baf-4deb-bfcf-59a1fff4a6a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3dfa40d3bc50431ba062dfe819ca58ca, user_id=7d0a16674f76445fb81eab0f76db806b, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:56.355382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60374f60-1baf-4deb-bfcf-59a1fff4a6a1 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:56.370722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 597/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:03:56 2022] PUT /identity/v3/projects/3dfa40d3bc50431ba062dfe819ca58ca/users/7d0a16674f76445fb81eab0f76db806b/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:03:58.386100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad116e-1633-48ed-a74e-20ac8bd52880 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:58.386675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad116e-1633-48ed-a74e-20ac8bd52880 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:58.386675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad116e-1633-48ed-a74e-20ac8bd52880 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:58.388804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 596/1193] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:03:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:03:58.397942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1449ca-81d9-4f98-9a44-67a4176c24fc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:58.398342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1449ca-81d9-4f98-9a44-67a4176c24fc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:58.398520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1449ca-81d9-4f98-9a44-67a4176c24fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:58.420032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2a1449ca-81d9-4f98-9a44-67a4176c24fc None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:03:58.421149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2a1449ca-81d9-4f98-9a44-67a4176c24fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:03:58.493126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 598/1194] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:03:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:03:58.516660 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad116e-1633-48ed-a74e-20ac8bd52880 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:58.558049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:58.626459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad116e-1633-48ed-a74e-20ac8bd52880 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:58.661487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee235ee-6612-4a13-bad0-0cd2406f0793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:58.662309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee235ee-6612-4a13-bad0-0cd2406f0793 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:58.662587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee235ee-6612-4a13-bad0-0cd2406f0793 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:58.662873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee235ee-6612-4a13-bad0-0cd2406f0793 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:58.674908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee235ee-6612-4a13-bad0-0cd2406f0793 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-7ee235ee-6612-4a13-bad0-0cd2406f0793 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:03:58.682715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:03:58.683801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:03:58.684676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 597/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:03:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:03:58.691924 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1449ca-81d9-4f98-9a44-67a4176c24fc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:58.804765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1449ca-81d9-4f98-9a44-67a4176c24fc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:58.813104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:58.842735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b656d-dd3d-4511-a151-2814360c9fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:58.843393 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b656d-dd3d-4511-a151-2814360c9fcf None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:58.843674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b656d-dd3d-4511-a151-2814360c9fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:58.843956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b656d-dd3d-4511-a151-2814360c9fcf None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:58.844704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b656d-dd3d-4511-a151-2814360c9fcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:58.845011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b656d-dd3d-4511-a151-2814360c9fcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:58.846441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b656d-dd3d-4511-a151-2814360c9fcf None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:58.854595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 599/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:03:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:58.860859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee235ee-6612-4a13-bad0-0cd2406f0793 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:58.958007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:58.964681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee235ee-6612-4a13-bad0-0cd2406f0793 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:59.004458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88adcf-0e24-41a8-a35d-ede1746d0ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:59.005357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88adcf-0e24-41a8-a35d-ede1746d0ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:59.005630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88adcf-0e24-41a8-a35d-ede1746d0ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:59.005926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88adcf-0e24-41a8-a35d-ede1746d0ca5 None admin] PATH_INFO: `/v3/users/7d0a16674f76445fb81eab0f76db806b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:59.007128 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88adcf-0e24-41a8-a35d-ede1746d0ca5 None admin] RBAC: Authorizing `identity:get_user(user_id=7d0a16674f76445fb81eab0f76db806b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:59.008557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88adcf-0e24-41a8-a35d-ede1746d0ca5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:59.010211 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 598/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 4 09:03:58 2022] GET /identity/v3/users/7d0a16674f76445fb81eab0f76db806b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:59.016229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b656d-dd3d-4511-a151-2814360c9fcf None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:59.123887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b656d-dd3d-4511-a151-2814360c9fcf None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:59.162796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06056376-8986-4782-8cdb-ed685635031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:59.163450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06056376-8986-4782-8cdb-ed685635031c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:59.163738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06056376-8986-4782-8cdb-ed685635031c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:59.164008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06056376-8986-4782-8cdb-ed685635031c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:59.201907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:59.245499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06056376-8986-4782-8cdb-ed685635031c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:59.246934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06056376-8986-4782-8cdb-ed685635031c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:59.357770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 600/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:03:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:03:59.364695 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88adcf-0e24-41a8-a35d-ede1746d0ca5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:59.478992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88adcf-0e24-41a8-a35d-ede1746d0ca5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:59.517193 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8873a6-46fe-4ad3-bd41-4de81ebb6461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:59.518006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8873a6-46fe-4ad3-bd41-4de81ebb6461 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:59.518295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8873a6-46fe-4ad3-bd41-4de81ebb6461 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:59.518570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8873a6-46fe-4ad3-bd41-4de81ebb6461 None admin] PATH_INFO: `/v3/projects/3dfa40d3bc50431ba062dfe819ca58ca` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:59.519849 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8873a6-46fe-4ad3-bd41-4de81ebb6461 None admin] RBAC: Authorizing `identity:get_project(project_id=3dfa40d3bc50431ba062dfe819ca58ca)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:59.521294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8873a6-46fe-4ad3-bd41-4de81ebb6461 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:59.523022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 599/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 4 09:03:59 2022] GET /identity/v3/projects/3dfa40d3bc50431ba062dfe819ca58ca => generated 361 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:03:59.529916 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06056376-8986-4782-8cdb-ed685635031c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:03:59.602701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:03:59.639252 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06056376-8986-4782-8cdb-ed685635031c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:03:59.676870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce997ce5-ad2e-4848-b1ea-8f982592bc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:03:59.677518 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce997ce5-ad2e-4848-b1ea-8f982592bc1e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:03:59.677801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce997ce5-ad2e-4848-b1ea-8f982592bc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:03:59.678115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce997ce5-ad2e-4848-b1ea-8f982592bc1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:03:59.678942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce997ce5-ad2e-4848-b1ea-8f982592bc1e None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=3dfa40d3bc50431ba062dfe819ca58ca, user.id=7d0a16674f76445fb81eab0f76db806b) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:03:59.679259 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce997ce5-ad2e-4848-b1ea-8f982592bc1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:03:59.680694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce997ce5-ad2e-4848-b1ea-8f982592bc1e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:03:59.687374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 601/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:03:59 2022] GET /identity/v3/role_assignments?user.id=7d0a16674f76445fb81eab0f76db806b&scope.project.id=3dfa40d3bc50431ba062dfe819ca58ca&role.id=2302517080ac42578aa81c606dd0ff2f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:04:09.509923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:04:09.684236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=34 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:04:54.960906 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af99d45-c89b-4e40-b6d7-47c4b7a8bfaa None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:54.961710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af99d45-c89b-4e40-b6d7-47c4b7a8bfaa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:54.961710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af99d45-c89b-4e40-b6d7-47c4b7a8bfaa None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:54.963346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 600/1201] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:04:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:04:54.968629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4802e-9989-49b9-a0ef-5222a61f6730 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:54.969120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4802e-9989-49b9-a0ef-5222a61f6730 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:54.969287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4802e-9989-49b9-a0ef-5222a61f6730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:54.989775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b3f4802e-9989-49b9-a0ef-5222a61f6730 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:04:54.991253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b3f4802e-9989-49b9-a0ef-5222a61f6730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:04:55.102435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 602/1202] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:04:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:04:55.130270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af99d45-c89b-4e40-b6d7-47c4b7a8bfaa None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:04:55.274219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af99d45-c89b-4e40-b6d7-47c4b7a8bfaa None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:04:55.316246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcddf1-72fa-4caf-be81-8b814250b70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:04:55.317163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcddf1-72fa-4caf-be81-8b814250b70d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:55.317461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcddf1-72fa-4caf-be81-8b814250b70d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:55.317731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcddf1-72fa-4caf-be81-8b814250b70d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:55.331436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcddf1-72fa-4caf-be81-8b814250b70d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:04:55.333374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcddf1-72fa-4caf-be81-8b814250b70d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-82fcddf1-72fa-4caf-be81-8b814250b70d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:04:55.354200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:04:55.355358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:04:55.356343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:04:55.356343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:04:55.356343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:04:55.356343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:04:55.356343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 601/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:04:55.361518 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4802e-9989-49b9-a0ef-5222a61f6730 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:04:55.500374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4802e-9989-49b9-a0ef-5222a61f6730 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:04:55.542714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:04:55.550411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4ceb4-ff2e-429a-b13e-c00a3f6953ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:04:55.551093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4ceb4-ff2e-429a-b13e-c00a3f6953ee None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:55.551378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4ceb4-ff2e-429a-b13e-c00a3f6953ee None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:55.551828 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4ceb4-ff2e-429a-b13e-c00a3f6953ee None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:55.552658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4ceb4-ff2e-429a-b13e-c00a3f6953ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:04:55.553069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4ceb4-ff2e-429a-b13e-c00a3f6953ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:04:55.554811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4ceb4-ff2e-429a-b13e-c00a3f6953ee None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:04:55.569073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 603/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:04:55.576937 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcddf1-72fa-4caf-be81-8b814250b70d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:04:55.650322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:04:55.708043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcddf1-72fa-4caf-be81-8b814250b70d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:04:55.756797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d26cd9-6928-42fd-bbbf-cf708a579939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:04:55.757641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d26cd9-6928-42fd-bbbf-cf708a579939 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:55.757952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d26cd9-6928-42fd-bbbf-cf708a579939 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:55.758233 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d26cd9-6928-42fd-bbbf-cf708a579939 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:55.758947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d26cd9-6928-42fd-bbbf-cf708a579939 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:04:55.761174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d26cd9-6928-42fd-bbbf-cf708a579939 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:04:55.789971 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 602/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 4 09:04:55 2022] POST /identity/v3/users => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:04:58.061652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aaa5b-684b-4813-98f7-3a0ec749818e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:58.062267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aaa5b-684b-4813-98f7-3a0ec749818e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:58.062267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aaa5b-684b-4813-98f7-3a0ec749818e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:58.063619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 604/1206] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:04:58.069435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d2c10-02af-4d2f-adf4-f4103681e26b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:58.069952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d2c10-02af-4d2f-adf4-f4103681e26b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:58.070044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d2c10-02af-4d2f-adf4-f4103681e26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:58.088305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e23d2c10-02af-4d2f-adf4-f4103681e26b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:04:58.089573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e23d2c10-02af-4d2f-adf4-f4103681e26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:04:58.167361 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 603/1207] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:04:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:04:58.191781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198aaa5b-684b-4813-98f7-3a0ec749818e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:04:58.312381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198aaa5b-684b-4813-98f7-3a0ec749818e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:04:58.351070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea9ede-d9f1-452d-b56c-454616f053d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:04:58.351879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea9ede-d9f1-452d-b56c-454616f053d4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:58.352158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea9ede-d9f1-452d-b56c-454616f053d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:58.352432 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea9ede-d9f1-452d-b56c-454616f053d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:58.364789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea9ede-d9f1-452d-b56c-454616f053d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-dfea9ede-d9f1-452d-b56c-454616f053d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:04:58.373204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:04:58.374608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:04:58.375729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 605/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:04:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:04:58.380871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d2c10-02af-4d2f-adf4-f4103681e26b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:04:58.430175 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:04:58.502709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d2c10-02af-4d2f-adf4-f4103681e26b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:04:58.547813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1146e6-b424-449e-9785-15e75e91188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:04:58.548527 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1146e6-b424-449e-9785-15e75e91188e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:58.548878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1146e6-b424-449e-9785-15e75e91188e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:58.549167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1146e6-b424-449e-9785-15e75e91188e None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:58.549997 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1146e6-b424-449e-9785-15e75e91188e None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:04:58.550427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1146e6-b424-449e-9785-15e75e91188e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:04:58.551934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1146e6-b424-449e-9785-15e75e91188e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:04:58.566942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 604/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:04:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:04:58.568476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea9ede-d9f1-452d-b56c-454616f053d4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:04:58.627931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:04:58.695338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea9ede-d9f1-452d-b56c-454616f053d4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:04:58.734923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cb792-e96b-470f-ab2a-36753885fb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:04:58.735797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb792-e96b-470f-ab2a-36753885fb0a None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:58.736084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb792-e96b-470f-ab2a-36753885fb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:58.736362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb792-e96b-470f-ab2a-36753885fb0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:58.762296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cb792-e96b-470f-ab2a-36753885fb0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:04:58.763769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43cb792-e96b-470f-ab2a-36753885fb0a None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-c43cb792-e96b-470f-ab2a-36753885fb0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:04:58.784554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:04:58.785623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:04:58.791566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:04:58.791566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:04:58.791566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:04:58.791566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:04:58.791566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 606/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:04:58.792072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1146e6-b424-449e-9785-15e75e91188e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:04:58.904369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:04:58.907572 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1146e6-b424-449e-9785-15e75e91188e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:04:58.948160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa813e9e-32f8-4f10-a2c3-eda62cf67dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:04:58.948873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa813e9e-32f8-4f10-a2c3-eda62cf67dec None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:58.949089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa813e9e-32f8-4f10-a2c3-eda62cf67dec None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:58.949390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa813e9e-32f8-4f10-a2c3-eda62cf67dec None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:58.950131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa813e9e-32f8-4f10-a2c3-eda62cf67dec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:04:58.950439 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa813e9e-32f8-4f10-a2c3-eda62cf67dec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:04:58.951841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa813e9e-32f8-4f10-a2c3-eda62cf67dec None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:04:58.969739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 605/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:04:58.971353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cb792-e96b-470f-ab2a-36753885fb0a None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:04:59.080475 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cb792-e96b-470f-ab2a-36753885fb0a None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:04:59.082072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:04:59.119714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10baa1-f86f-4486-a71c-97396e7e48c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:04:59.120538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10baa1-f86f-4486-a71c-97396e7e48c0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:59.120807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10baa1-f86f-4486-a71c-97396e7e48c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:59.121076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10baa1-f86f-4486-a71c-97396e7e48c0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:59.129325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10baa1-f86f-4486-a71c-97396e7e48c0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:04:59.131444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10baa1-f86f-4486-a71c-97396e7e48c0 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5f10baa1-f86f-4486-a71c-97396e7e48c0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:04:59.146397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:04:59.147538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:04:59.148596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:04:59.148596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:04:59.148596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:04:59.148596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:04:59.148596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:04:59.148596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:04:59.148596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:04:59.148596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:04:59.148596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:04:59.148596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:04:59.148596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 607/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:04:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:04:59.154004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa813e9e-32f8-4f10-a2c3-eda62cf67dec None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:04:59.260868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa813e9e-32f8-4f10-a2c3-eda62cf67dec None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:04:59.295759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d38c9f-05cc-4c80-8b4e-32faba0988a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:04:59.296440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d38c9f-05cc-4c80-8b4e-32faba0988a5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:59.296723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d38c9f-05cc-4c80-8b4e-32faba0988a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:59.297006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d38c9f-05cc-4c80-8b4e-32faba0988a5 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:59.297854 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d38c9f-05cc-4c80-8b4e-32faba0988a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:04:59.298191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d38c9f-05cc-4c80-8b4e-32faba0988a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:04:59.299624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d38c9f-05cc-4c80-8b4e-32faba0988a5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:04:59.322128 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 606/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 4 09:04:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:04:59.323386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10baa1-f86f-4486-a71c-97396e7e48c0 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:04:59.439727 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10baa1-f86f-4486-a71c-97396e7e48c0 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:04:59.478625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ed035-5e5d-42dd-a0dd-2a2265798c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:04:59.479444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ed035-5e5d-42dd-a0dd-2a2265798c24 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:59.479819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ed035-5e5d-42dd-a0dd-2a2265798c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:59.480171 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ed035-5e5d-42dd-a0dd-2a2265798c24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:59.513714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:04:59.561136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ed035-5e5d-42dd-a0dd-2a2265798c24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:04:59.562695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ed035-5e5d-42dd-a0dd-2a2265798c24 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:04:59.684474 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 608/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:04:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:04:59.698753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d38c9f-05cc-4c80-8b4e-32faba0988a5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:04:59.726144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:04:59.834533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d38c9f-05cc-4c80-8b4e-32faba0988a5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:04:59.892491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194a373-70ed-4c6b-b005-7396d1f0bd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:04:59.893316 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194a373-70ed-4c6b-b005-7396d1f0bd3c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:04:59.893605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194a373-70ed-4c6b-b005-7396d1f0bd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:04:59.893938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194a373-70ed-4c6b-b005-7396d1f0bd3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:04:59.905512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194a373-70ed-4c6b-b005-7396d1f0bd3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:04:59.907007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194a373-70ed-4c6b-b005-7396d1f0bd3c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8194a373-70ed-4c6b-b005-7396d1f0bd3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:04:59.927440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:04:59.929158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:04:59.930332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:04:59.930332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:04:59.930332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:04:59.930332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:04:59.930332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 607/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:04:59.934979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ed035-5e5d-42dd-a0dd-2a2265798c24 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:00.009514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:00.052734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ed035-5e5d-42dd-a0dd-2a2265798c24 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:00.095620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70697c-c446-4da6-8222-35ff48895137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:00.096574 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70697c-c446-4da6-8222-35ff48895137 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:00.096862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70697c-c446-4da6-8222-35ff48895137 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:00.097212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70697c-c446-4da6-8222-35ff48895137 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:00.098450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70697c-c446-4da6-8222-35ff48895137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:00.098763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70697c-c446-4da6-8222-35ff48895137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:00.100548 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70697c-c446-4da6-8222-35ff48895137 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:00.117916 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 609/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:00.120881 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194a373-70ed-4c6b-b005-7396d1f0bd3c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:00.216813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:00.262605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194a373-70ed-4c6b-b005-7396d1f0bd3c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:00.301254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdbd2d-2fb6-41bc-afa8-c9b9d5880542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:00.302098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdbd2d-2fb6-41bc-afa8-c9b9d5880542 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:00.302380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdbd2d-2fb6-41bc-afa8-c9b9d5880542 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:00.302651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdbd2d-2fb6-41bc-afa8-c9b9d5880542 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:00.309374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fdbd2d-2fb6-41bc-afa8-c9b9d5880542 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:00.310843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fdbd2d-2fb6-41bc-afa8-c9b9d5880542 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-84fdbd2d-2fb6-41bc-afa8-c9b9d5880542 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:05:00.325804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:00.327002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:00.327995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:05:00.327995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:05:00.327995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:05:00.327995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:05:00.327995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:05:00.327995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:05:00.327995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:00.327995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:00.327995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 608/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:05:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:00.335176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70697c-c446-4da6-8222-35ff48895137 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:00.450397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70697c-c446-4da6-8222-35ff48895137 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:00.475791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:00.491021 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456ab7e-1e52-499b-8d66-deefe5921034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:00.491838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456ab7e-1e52-499b-8d66-deefe5921034 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:00.492156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456ab7e-1e52-499b-8d66-deefe5921034 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:00.492412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456ab7e-1e52-499b-8d66-deefe5921034 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:00.493289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456ab7e-1e52-499b-8d66-deefe5921034 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:00.493607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456ab7e-1e52-499b-8d66-deefe5921034 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:00.495265 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456ab7e-1e52-499b-8d66-deefe5921034 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:00.508764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 610/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:05:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:00.515088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdbd2d-2fb6-41bc-afa8-c9b9d5880542 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:00.633283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdbd2d-2fb6-41bc-afa8-c9b9d5880542 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:00.664975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:00.672634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4d255-042e-43ff-82dd-4d012f2da953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:00.673376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4d255-042e-43ff-82dd-4d012f2da953 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:00.673733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4d255-042e-43ff-82dd-4d012f2da953 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:00.674103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4d255-042e-43ff-82dd-4d012f2da953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:00.675007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4d255-042e-43ff-82dd-4d012f2da953 None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=aff2305249a242639e1415094bf9df33) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:00.675396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4d255-042e-43ff-82dd-4d012f2da953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:00.677066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4d255-042e-43ff-82dd-4d012f2da953 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:00.683616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 609/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:00 2022] GET /identity/v3/role_assignments?user.id=aff2305249a242639e1415094bf9df33&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:02.919761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934ee3-5e99-4eec-b8b7-5857ea015666 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:02.920336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934ee3-5e99-4eec-b8b7-5857ea015666 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:02.920336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934ee3-5e99-4eec-b8b7-5857ea015666 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:02.921671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 611/1220] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:02.927749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536f47-78af-4787-8a9e-7abfb8906044 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:02.928176 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536f47-78af-4787-8a9e-7abfb8906044 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:02.928324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536f47-78af-4787-8a9e-7abfb8906044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:02.947154 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-34536f47-78af-4787-8a9e-7abfb8906044 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:02.948602 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-34536f47-78af-4787-8a9e-7abfb8906044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:03.031103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 610/1221] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:05:03.054589 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934ee3-5e99-4eec-b8b7-5857ea015666 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:03.177835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934ee3-5e99-4eec-b8b7-5857ea015666 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:03.214375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe5ea-c87f-4081-83ef-23815b5c8020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:03.215166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe5ea-c87f-4081-83ef-23815b5c8020 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:03.215451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe5ea-c87f-4081-83ef-23815b5c8020 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:03.215728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe5ea-c87f-4081-83ef-23815b5c8020 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:03.226357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fe5ea-c87f-4081-83ef-23815b5c8020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ba7fe5ea-c87f-4081-83ef-23815b5c8020 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:03.234195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:03.235399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:03.236448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 612/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:05:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:03.242444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34536f47-78af-4787-8a9e-7abfb8906044 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:03.352233 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34536f47-78af-4787-8a9e-7abfb8906044 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:03.379132 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:03.391031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27cc7f-d069-42fd-9c27-204f7dd57663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:03.391707 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27cc7f-d069-42fd-9c27-204f7dd57663 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:03.391992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27cc7f-d069-42fd-9c27-204f7dd57663 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:03.392267 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27cc7f-d069-42fd-9c27-204f7dd57663 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:03.393027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27cc7f-d069-42fd-9c27-204f7dd57663 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:03.393333 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27cc7f-d069-42fd-9c27-204f7dd57663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:03.394745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27cc7f-d069-42fd-9c27-204f7dd57663 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:03.403405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 611/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:05:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:03.409751 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe5ea-c87f-4081-83ef-23815b5c8020 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:03.458130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:03.520970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe5ea-c87f-4081-83ef-23815b5c8020 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:03.561764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8aeefd-10da-4dca-b5a1-cc36d54294eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:03.562595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aeefd-10da-4dca-b5a1-cc36d54294eb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:03.562880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aeefd-10da-4dca-b5a1-cc36d54294eb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:03.563156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aeefd-10da-4dca-b5a1-cc36d54294eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:03.573803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8aeefd-10da-4dca-b5a1-cc36d54294eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:03.575310 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8aeefd-10da-4dca-b5a1-cc36d54294eb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5b8aeefd-10da-4dca-b5a1-cc36d54294eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:03.594540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:03.595533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:03.596780 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:03.596780 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:03.596780 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:03.596780 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:03.596780 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 613/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:03.602877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27cc7f-d069-42fd-9c27-204f7dd57663 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:03.712965 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27cc7f-d069-42fd-9c27-204f7dd57663 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:03.748584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f893e-d27d-465c-9b3f-61b42c5b77de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:03.749289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f893e-d27d-465c-9b3f-61b42c5b77de None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:03.749599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f893e-d27d-465c-9b3f-61b42c5b77de None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:03.749877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f893e-d27d-465c-9b3f-61b42c5b77de None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:03.750754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f893e-d27d-465c-9b3f-61b42c5b77de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:03.751068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f893e-d27d-465c-9b3f-61b42c5b77de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:03.752720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f893e-d27d-465c-9b3f-61b42c5b77de None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:03.764772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 612/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:03.770767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8aeefd-10da-4dca-b5a1-cc36d54294eb None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:03.896212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:03.904144 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8aeefd-10da-4dca-b5a1-cc36d54294eb None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:03.943439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baffe47-75b8-48d9-a7ae-00597485a58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:03.944253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baffe47-75b8-48d9-a7ae-00597485a58c None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:03.944670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baffe47-75b8-48d9-a7ae-00597485a58c None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:03.944963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baffe47-75b8-48d9-a7ae-00597485a58c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:03.953720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baffe47-75b8-48d9-a7ae-00597485a58c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:03.955339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baffe47-75b8-48d9-a7ae-00597485a58c None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-2baffe47-75b8-48d9-a7ae-00597485a58c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:05:03.970818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:03.971817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:03.973136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:03.973136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:03.973136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:05:03.973136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:05:03.973136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:05:03.973136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:05:03.973136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:05:03.973136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:05:03.973136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:05:03.973136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:03.973136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 614/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:03.977993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f893e-d27d-465c-9b3f-61b42c5b77de None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:04.029740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:04.088556 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f893e-d27d-465c-9b3f-61b42c5b77de None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:04.126130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2bb0b-32cb-496c-b3a1-6fd6fbec5b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:04.126816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2bb0b-32cb-496c-b3a1-6fd6fbec5b06 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:04.127103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2bb0b-32cb-496c-b3a1-6fd6fbec5b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:04.127381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2bb0b-32cb-496c-b3a1-6fd6fbec5b06 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:04.128329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2bb0b-32cb-496c-b3a1-6fd6fbec5b06 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:04.128786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2bb0b-32cb-496c-b3a1-6fd6fbec5b06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:04.130909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2bb0b-32cb-496c-b3a1-6fd6fbec5b06 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:04.153917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 613/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 4 09:05:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:04.158236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baffe47-75b8-48d9-a7ae-00597485a58c None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:04.281511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baffe47-75b8-48d9-a7ae-00597485a58c None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:04.317334 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29f0a8-7c90-4046-8f58-a4173a26dbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:04.318243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f0a8-7c90-4046-8f58-a4173a26dbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:04.318533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f0a8-7c90-4046-8f58-a4173a26dbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:04.318807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f0a8-7c90-4046-8f58-a4173a26dbd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:04.333939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29f0a8-7c90-4046-8f58-a4173a26dbd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:04.335494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29f0a8-7c90-4046-8f58-a4173a26dbd9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-9e29f0a8-7c90-4046-8f58-a4173a26dbd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:04.353278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:04.354757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:04.356004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:04.356004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:04.356004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:04.356004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:04.356004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 615/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:04.360399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2bb0b-32cb-496c-b3a1-6fd6fbec5b06 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:04.464930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2bb0b-32cb-496c-b3a1-6fd6fbec5b06 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:04.467364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:04.502539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1c88-3afe-4e8f-85cc-f6fe2f843af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:04.503204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1c88-3afe-4e8f-85cc-f6fe2f843af5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:04.503485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1c88-3afe-4e8f-85cc-f6fe2f843af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:04.503757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1c88-3afe-4e8f-85cc-f6fe2f843af5 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:04.504617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a1c88-3afe-4e8f-85cc-f6fe2f843af5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:04.504924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a1c88-3afe-4e8f-85cc-f6fe2f843af5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:04.506643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a1c88-3afe-4e8f-85cc-f6fe2f843af5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:04.518592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 614/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:04.524517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29f0a8-7c90-4046-8f58-a4173a26dbd9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:04.535035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:04.643847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29f0a8-7c90-4046-8f58-a4173a26dbd9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:04.680514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f805467-15aa-4bd9-9f74-4fdf0498695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:04.681434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805467-15aa-4bd9-9f74-4fdf0498695f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:04.681721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805467-15aa-4bd9-9f74-4fdf0498695f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:04.682085 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805467-15aa-4bd9-9f74-4fdf0498695f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:04.690274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f805467-15aa-4bd9-9f74-4fdf0498695f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:04.691914 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f805467-15aa-4bd9-9f74-4fdf0498695f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5f805467-15aa-4bd9-9f74-4fdf0498695f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:05:04.708130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:04.709355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:04.710495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:05:04.710495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:05:04.710495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:05:04.710495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:05:04.710495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:05:04.710495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:05:04.710495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:04.710495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:04.710495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 616/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:05:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:04.715516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1c88-3afe-4e8f-85cc-f6fe2f843af5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:04.820520 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1c88-3afe-4e8f-85cc-f6fe2f843af5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:04.855940 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a60964-56c4-4951-9014-a04aa1e39c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:04.856667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a60964-56c4-4951-9014-a04aa1e39c07 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:04.856949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a60964-56c4-4951-9014-a04aa1e39c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:04.857223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a60964-56c4-4951-9014-a04aa1e39c07 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:04.858099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a60964-56c4-4951-9014-a04aa1e39c07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:04.858431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a60964-56c4-4951-9014-a04aa1e39c07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:04.859974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a60964-56c4-4951-9014-a04aa1e39c07 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:04.873634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 615/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:05:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:04.879761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f805467-15aa-4bd9-9f74-4fdf0498695f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:04.952528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:04.988539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f805467-15aa-4bd9-9f74-4fdf0498695f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:05.025761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996836e-6bd2-49fa-9e70-d0533c43abb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:05.026703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996836e-6bd2-49fa-9e70-d0533c43abb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:05.026983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996836e-6bd2-49fa-9e70-d0533c43abb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:05.027254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996836e-6bd2-49fa-9e70-d0533c43abb3 None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/aff2305249a242639e1415094bf9df33/roles/f0e6d5b558954227a2e1423c3d1e5d18` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:05.043372 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996836e-6bd2-49fa-9e70-d0533c43abb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=aff2305249a242639e1415094bf9df33, role_id=f0e6d5b558954227a2e1423c3d1e5d18)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:05.045008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996836e-6bd2-49fa-9e70-d0533c43abb3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:05.060156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 617/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:05:04 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/aff2305249a242639e1415094bf9df33/roles/f0e6d5b558954227a2e1423c3d1e5d18 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:05:07.119824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f48dd-5f6d-40fa-98d1-f0d3a49bca1c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:07.120401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f48dd-5f6d-40fa-98d1-f0d3a49bca1c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:07.120401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f48dd-5f6d-40fa-98d1-f0d3a49bca1c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:07.122141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 616/1233] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:07.127361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0d252-e89b-471e-8c60-bc2556520661 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:07.127811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0d252-e89b-471e-8c60-bc2556520661 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:07.127905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0d252-e89b-471e-8c60-bc2556520661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:07.147042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1fe0d252-e89b-471e-8c60-bc2556520661 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:07.147335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1fe0d252-e89b-471e-8c60-bc2556520661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:07.215508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 618/1234] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:05:07.240700 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f48dd-5f6d-40fa-98d1-f0d3a49bca1c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:07.268622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:07.354237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f48dd-5f6d-40fa-98d1-f0d3a49bca1c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:07.389124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a3073-ddbd-4d65-911c-838c26d4bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:07.389847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a3073-ddbd-4d65-911c-838c26d4bfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:07.390157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a3073-ddbd-4d65-911c-838c26d4bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:07.390502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a3073-ddbd-4d65-911c-838c26d4bfc2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:07.403770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a3073-ddbd-4d65-911c-838c26d4bfc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9c6a3073-ddbd-4d65-911c-838c26d4bfc2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:07.414502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:07.415534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 04 09:05:07.416480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:07.417131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 617/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 4 09:05:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:07.422602 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0d252-e89b-471e-8c60-bc2556520661 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:07.481564 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:07.537923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0d252-e89b-471e-8c60-bc2556520661 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:07.575349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb26dd-b59c-4142-a0f7-5721334d4db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:07.576033 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb26dd-b59c-4142-a0f7-5721334d4db0 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:07.576316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb26dd-b59c-4142-a0f7-5721334d4db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:07.576591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb26dd-b59c-4142-a0f7-5721334d4db0 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:07.577339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb26dd-b59c-4142-a0f7-5721334d4db0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:07.577644 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb26dd-b59c-4142-a0f7-5721334d4db0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:07.579824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb26dd-b59c-4142-a0f7-5721334d4db0 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:07.591006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 619/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 4 09:05:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:07.600357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a3073-ddbd-4d65-911c-838c26d4bfc2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:07.708198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:07.726184 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a3073-ddbd-4d65-911c-838c26d4bfc2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:07.762944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3993e-52ae-466d-b531-8c097cf83591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:07.763744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3993e-52ae-466d-b531-8c097cf83591 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:07.764072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3993e-52ae-466d-b531-8c097cf83591 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:07.764354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3993e-52ae-466d-b531-8c097cf83591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:07.776214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3993e-52ae-466d-b531-8c097cf83591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:07.777678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3993e-52ae-466d-b531-8c097cf83591 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-32e3993e-52ae-466d-b531-8c097cf83591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:07.795052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:07.796203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:07.797809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:07.797809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:07.797809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:07.797809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:07.797809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 618/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:07.802237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb26dd-b59c-4142-a0f7-5721334d4db0 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:07.912797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb26dd-b59c-4142-a0f7-5721334d4db0 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:07.929595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:07.966258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8633f5-715b-4794-9ee7-6f911cf054a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:07.966859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8633f5-715b-4794-9ee7-6f911cf054a8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:07.967213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8633f5-715b-4794-9ee7-6f911cf054a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:07.967430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8633f5-715b-4794-9ee7-6f911cf054a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:07.968174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8633f5-715b-4794-9ee7-6f911cf054a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:07.968485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8633f5-715b-4794-9ee7-6f911cf054a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:07.969866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8633f5-715b-4794-9ee7-6f911cf054a8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:07.982821 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 620/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:07.988515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3993e-52ae-466d-b531-8c097cf83591 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:08.104425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3993e-52ae-466d-b531-8c097cf83591 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:08.143389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1299ea-a21c-4a02-85cc-67d22e24aec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:08.144293 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1299ea-a21c-4a02-85cc-67d22e24aec7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:08.144572 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1299ea-a21c-4a02-85cc-67d22e24aec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:08.144860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1299ea-a21c-4a02-85cc-67d22e24aec7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:08.147815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:08.153480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1299ea-a21c-4a02-85cc-67d22e24aec7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:08.155272 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1299ea-a21c-4a02-85cc-67d22e24aec7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-fa1299ea-a21c-4a02-85cc-67d22e24aec7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:05:08.170474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:08.171459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:08.173136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:08.173136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:08.173136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:05:08.173136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:05:08.173136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:05:08.173136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:05:08.173136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:05:08.173136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:05:08.173136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:05:08.173136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:08.173136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 619/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:08.177756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8633f5-715b-4794-9ee7-6f911cf054a8 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:08.283991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8633f5-715b-4794-9ee7-6f911cf054a8 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:08.326092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc35fac-5c45-416c-8f6a-d337be918fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:08.326998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc35fac-5c45-416c-8f6a-d337be918fdd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:08.327410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc35fac-5c45-416c-8f6a-d337be918fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:08.327749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc35fac-5c45-416c-8f6a-d337be918fdd None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:08.328943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc35fac-5c45-416c-8f6a-d337be918fdd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:08.329255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc35fac-5c45-416c-8f6a-d337be918fdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:08.331246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc35fac-5c45-416c-8f6a-d337be918fdd None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:08.353496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 621/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 4 09:05:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:08.360364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1299ea-a21c-4a02-85cc-67d22e24aec7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:08.488036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1299ea-a21c-4a02-85cc-67d22e24aec7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:08.530773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d483071-57d2-49ae-be06-da469e9c300d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:08.531091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d483071-57d2-49ae-be06-da469e9c300d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:08.531379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d483071-57d2-49ae-be06-da469e9c300d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:08.531697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d483071-57d2-49ae-be06-da469e9c300d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:08.600846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:08.608324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d483071-57d2-49ae-be06-da469e9c300d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:08.610048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d483071-57d2-49ae-be06-da469e9c300d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:08.716423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 620/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:05:08.722342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc35fac-5c45-416c-8f6a-d337be918fdd None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:08.753581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:08.831065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc35fac-5c45-416c-8f6a-d337be918fdd None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:08.867302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a81be-fee4-4aaf-b0b7-f34257cd742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:08.868003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a81be-fee4-4aaf-b0b7-f34257cd742b None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:08.868286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a81be-fee4-4aaf-b0b7-f34257cd742b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:08.868565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a81be-fee4-4aaf-b0b7-f34257cd742b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:08.880516 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a81be-fee4-4aaf-b0b7-f34257cd742b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:08.882294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a81be-fee4-4aaf-b0b7-f34257cd742b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-929a81be-fee4-4aaf-b0b7-f34257cd742b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:08.905424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:08.906632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:08.912511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:08.912511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:08.912511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:08.912511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:08.912511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 622/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:08.913417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d483071-57d2-49ae-be06-da469e9c300d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:09.019989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:09.026726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d483071-57d2-49ae-be06-da469e9c300d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:09.066262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587afab-f998-4d36-b050-6b6852b380f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:09.067220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587afab-f998-4d36-b050-6b6852b380f8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:09.067712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587afab-f998-4d36-b050-6b6852b380f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:09.068019 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587afab-f998-4d36-b050-6b6852b380f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:09.068888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587afab-f998-4d36-b050-6b6852b380f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:09.069221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587afab-f998-4d36-b050-6b6852b380f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:09.070673 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587afab-f998-4d36-b050-6b6852b380f8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:09.083486 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 621/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:09.089451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a81be-fee4-4aaf-b0b7-f34257cd742b None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:09.182277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:09.205102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a81be-fee4-4aaf-b0b7-f34257cd742b None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:09.240712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8579715-90d6-49c8-85b5-6de699d82f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:09.241492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8579715-90d6-49c8-85b5-6de699d82f2f None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:09.241825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8579715-90d6-49c8-85b5-6de699d82f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:09.242130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8579715-90d6-49c8-85b5-6de699d82f2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:09.249764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8579715-90d6-49c8-85b5-6de699d82f2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:09.251463 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8579715-90d6-49c8-85b5-6de699d82f2f None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-b8579715-90d6-49c8-85b5-6de699d82f2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:05:09.266697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:09.267817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:09.268845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:05:09.268845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:05:09.268845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:05:09.268845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:05:09.268845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:05:09.268845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:05:09.268845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:09.268845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:09.268845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 623/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:05:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:09.274004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587afab-f998-4d36-b050-6b6852b380f8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:09.378382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587afab-f998-4d36-b050-6b6852b380f8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:09.412931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2a414-e8a4-4d6e-a6f6-d1d9ac01b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:09.413532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2a414-e8a4-4d6e-a6f6-d1d9ac01b35c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:09.413816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2a414-e8a4-4d6e-a6f6-d1d9ac01b35c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:09.414125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2a414-e8a4-4d6e-a6f6-d1d9ac01b35c None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:09.414924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2a414-e8a4-4d6e-a6f6-d1d9ac01b35c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:09.415236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2a414-e8a4-4d6e-a6f6-d1d9ac01b35c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:09.416674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2a414-e8a4-4d6e-a6f6-d1d9ac01b35c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:09.429309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 622/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:05:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:09.435358 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8579715-90d6-49c8-85b5-6de699d82f2f None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:09.550659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8579715-90d6-49c8-85b5-6de699d82f2f None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:09.589748 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafe10b-9935-4069-8590-0459888bbec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:09.590417 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafe10b-9935-4069-8590-0459888bbec6 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:09.590736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafe10b-9935-4069-8590-0459888bbec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:09.590970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafe10b-9935-4069-8590-0459888bbec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:09.591878 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafe10b-9935-4069-8590-0459888bbec6 None admin] RBAC: Adding query filter params (role.id=f0e6d5b558954227a2e1423c3d1e5d18, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=aff2305249a242639e1415094bf9df33) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:09.592189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafe10b-9935-4069-8590-0459888bbec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:09.593621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafe10b-9935-4069-8590-0459888bbec6 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:09.599848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 624/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:09 2022] GET /identity/v3/role_assignments?user.id=aff2305249a242639e1415094bf9df33&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=f0e6d5b558954227a2e1423c3d1e5d18 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:11.673058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88bf6-f76d-4645-901f-1f6f366fd8de None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:11.673696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88bf6-f76d-4645-901f-1f6f366fd8de None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:11.673696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88bf6-f76d-4645-901f-1f6f366fd8de None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:11.675294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 623/1247] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:05:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:11.680580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302923d-89e8-4c16-9667-506709e4af9e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:11.681022 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302923d-89e8-4c16-9667-506709e4af9e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:11.681147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302923d-89e8-4c16-9667-506709e4af9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:11.687336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:11.699718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d302923d-89e8-4c16-9667-506709e4af9e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:11.700919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d302923d-89e8-4c16-9667-506709e4af9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:11.770944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 625/1248] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:05:11.795027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c88bf6-f76d-4645-901f-1f6f366fd8de None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:11.815063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:11.910499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c88bf6-f76d-4645-901f-1f6f366fd8de None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:11.958632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0aa198-3c96-402e-b614-7bf83ab7bba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:11.959354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aa198-3c96-402e-b614-7bf83ab7bba7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:11.959785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aa198-3c96-402e-b614-7bf83ab7bba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:11.960066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aa198-3c96-402e-b614-7bf83ab7bba7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:11.971558 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0aa198-3c96-402e-b614-7bf83ab7bba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-4b0aa198-3c96-402e-b614-7bf83ab7bba7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:11.979436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:11.980640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:11.981513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 624/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:11.986797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302923d-89e8-4c16-9667-506709e4af9e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:12.093614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302923d-89e8-4c16-9667-506709e4af9e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:12.126977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:12.132213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e507df-fa6f-474b-b5a7-d4b8ba04d154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:12.132899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e507df-fa6f-474b-b5a7-d4b8ba04d154 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:12.133183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e507df-fa6f-474b-b5a7-d4b8ba04d154 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:12.133476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e507df-fa6f-474b-b5a7-d4b8ba04d154 None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:12.134278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e507df-fa6f-474b-b5a7-d4b8ba04d154 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:12.134597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e507df-fa6f-474b-b5a7-d4b8ba04d154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:12.136400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e507df-fa6f-474b-b5a7-d4b8ba04d154 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:12.145216 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 626/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:12.151110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0aa198-3c96-402e-b614-7bf83ab7bba7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:12.231837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:12.259295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0aa198-3c96-402e-b614-7bf83ab7bba7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:12.294261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466529f7-bde2-4b80-b02a-0c8e582ab54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:12.295033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466529f7-bde2-4b80-b02a-0c8e582ab54f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:12.295323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466529f7-bde2-4b80-b02a-0c8e582ab54f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:12.295614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466529f7-bde2-4b80-b02a-0c8e582ab54f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:12.306575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466529f7-bde2-4b80-b02a-0c8e582ab54f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:12.308032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466529f7-bde2-4b80-b02a-0c8e582ab54f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-466529f7-bde2-4b80-b02a-0c8e582ab54f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:12.325133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:12.326121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:12.327048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:12.327048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:12.327048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:12.327048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:12.327048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 625/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:12.332445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e507df-fa6f-474b-b5a7-d4b8ba04d154 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:12.445070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e507df-fa6f-474b-b5a7-d4b8ba04d154 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:12.482207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bc056-b7bd-41c4-8c0f-46e7bfa297cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:12.482845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc056-b7bd-41c4-8c0f-46e7bfa297cf None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:12.483132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc056-b7bd-41c4-8c0f-46e7bfa297cf None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:12.483548 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc056-b7bd-41c4-8c0f-46e7bfa297cf None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:12.484381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bc056-b7bd-41c4-8c0f-46e7bfa297cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:12.484711 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bc056-b7bd-41c4-8c0f-46e7bfa297cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:12.487008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bc056-b7bd-41c4-8c0f-46e7bfa297cf None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:12.503325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 627/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:12.509579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466529f7-bde2-4b80-b02a-0c8e582ab54f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:12.622050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466529f7-bde2-4b80-b02a-0c8e582ab54f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:12.658152 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:12.667823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032ecc7-49f7-4fc2-afff-4636b457de4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:12.668638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032ecc7-49f7-4fc2-afff-4636b457de4f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:12.668927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032ecc7-49f7-4fc2-afff-4636b457de4f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:12.669199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032ecc7-49f7-4fc2-afff-4636b457de4f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:12.679121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032ecc7-49f7-4fc2-afff-4636b457de4f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:12.680594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032ecc7-49f7-4fc2-afff-4636b457de4f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-7032ecc7-49f7-4fc2-afff-4636b457de4f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:05:12.695911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:12.696964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:12.698764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:12.698764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:12.698764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:05:12.698764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:05:12.698764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:05:12.698764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:05:12.698764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:05:12.698764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:05:12.698764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:05:12.698764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:12.698764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 626/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:12.703121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bc056-b7bd-41c4-8c0f-46e7bfa297cf None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:12.766511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:12.813299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bc056-b7bd-41c4-8c0f-46e7bfa297cf None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:12.849275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5536ec8-b5ae-4a15-a4f4-bd21a4ba57d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:12.849983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5536ec8-b5ae-4a15-a4f4-bd21a4ba57d9 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:12.850296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5536ec8-b5ae-4a15-a4f4-bd21a4ba57d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:12.850546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5536ec8-b5ae-4a15-a4f4-bd21a4ba57d9 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:12.851416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5536ec8-b5ae-4a15-a4f4-bd21a4ba57d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:12.851714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5536ec8-b5ae-4a15-a4f4-bd21a4ba57d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:12.853149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5536ec8-b5ae-4a15-a4f4-bd21a4ba57d9 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:12.871833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 628/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 4 09:05:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:12.878139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032ecc7-49f7-4fc2-afff-4636b457de4f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:12.991550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032ecc7-49f7-4fc2-afff-4636b457de4f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:13.027208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1db61-b864-4d3f-b7a9-b1daddc4a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:13.027803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1db61-b864-4d3f-b7a9-b1daddc4a3df None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:13.028086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1db61-b864-4d3f-b7a9-b1daddc4a3df None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:13.028376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1db61-b864-4d3f-b7a9-b1daddc4a3df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:13.085453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:13.098288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1db61-b864-4d3f-b7a9-b1daddc4a3df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:13.099753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1db61-b864-4d3f-b7a9-b1daddc4a3df None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:13.209131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 627/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 04 09:05:13.214968 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5536ec8-b5ae-4a15-a4f4-bd21a4ba57d9 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:13.327068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5536ec8-b5ae-4a15-a4f4-bd21a4ba57d9 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:13.340883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:13.367122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b9d02-9e5b-4901-a68b-2b222392be27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:13.367926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b9d02-9e5b-4901-a68b-2b222392be27 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:13.368201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b9d02-9e5b-4901-a68b-2b222392be27 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:13.368477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b9d02-9e5b-4901-a68b-2b222392be27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:13.379207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b9d02-9e5b-4901-a68b-2b222392be27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:13.380761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b9d02-9e5b-4901-a68b-2b222392be27 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-e14b9d02-9e5b-4901-a68b-2b222392be27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:13.398601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:13.399588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:13.400714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:13.400714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:13.400714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:13.400714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:13.400714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 629/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:13.405595 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1db61-b864-4d3f-b7a9-b1daddc4a3df None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:13.497767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:13.512107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1db61-b864-4d3f-b7a9-b1daddc4a3df None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:13.551362 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e0b58-e5ee-41ef-a7de-787d8f2f85e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:13.552070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e0b58-e5ee-41ef-a7de-787d8f2f85e6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:13.552356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e0b58-e5ee-41ef-a7de-787d8f2f85e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:13.552628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e0b58-e5ee-41ef-a7de-787d8f2f85e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:13.553350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e0b58-e5ee-41ef-a7de-787d8f2f85e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:13.553657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e0b58-e5ee-41ef-a7de-787d8f2f85e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:13.555097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e0b58-e5ee-41ef-a7de-787d8f2f85e6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:13.566733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 628/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:13.572477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b9d02-9e5b-4901-a68b-2b222392be27 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:13.682727 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b9d02-9e5b-4901-a68b-2b222392be27 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:13.716583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bca43-6b4b-40a2-9fb5-e43c4ea7d5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:13.717394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bca43-6b4b-40a2-9fb5-e43c4ea7d5ee None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:13.717680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bca43-6b4b-40a2-9fb5-e43c4ea7d5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:13.717990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bca43-6b4b-40a2-9fb5-e43c4ea7d5ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:13.724998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bca43-6b4b-40a2-9fb5-e43c4ea7d5ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:13.726501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bca43-6b4b-40a2-9fb5-e43c4ea7d5ee None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-ea3bca43-6b4b-40a2-9fb5-e43c4ea7d5ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:05:13.741056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:13.742199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:13.743380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:05:13.743380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:05:13.743380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:05:13.743380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:05:13.743380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:05:13.743380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:05:13.743380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:13.743380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:13.743380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 630/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:05:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:13.748276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e0b58-e5ee-41ef-a7de-787d8f2f85e6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:13.865579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e0b58-e5ee-41ef-a7de-787d8f2f85e6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:13.904606 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b74623-10e6-4af6-ad0c-ac8df4acb34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:13.905291 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b74623-10e6-4af6-ad0c-ac8df4acb34f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:13.905578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b74623-10e6-4af6-ad0c-ac8df4acb34f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:13.905976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b74623-10e6-4af6-ad0c-ac8df4acb34f None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:13.906704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b74623-10e6-4af6-ad0c-ac8df4acb34f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:13.907013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b74623-10e6-4af6-ad0c-ac8df4acb34f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:13.908486 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b74623-10e6-4af6-ad0c-ac8df4acb34f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:13.923041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 629/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:13.929975 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bca43-6b4b-40a2-9fb5-e43c4ea7d5ee None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:13.965141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:14.064798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bca43-6b4b-40a2-9fb5-e43c4ea7d5ee None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:14.104333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a05f5-2065-419e-aed6-4b9e98ac156e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:14.105244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a05f5-2065-419e-aed6-4b9e98ac156e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:14.105534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a05f5-2065-419e-aed6-4b9e98ac156e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:14.105816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a05f5-2065-419e-aed6-4b9e98ac156e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:14.106768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a05f5-2065-419e-aed6-4b9e98ac156e None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=aff2305249a242639e1415094bf9df33) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:14.107139 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a05f5-2065-419e-aed6-4b9e98ac156e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:14.108774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a05f5-2065-419e-aed6-4b9e98ac156e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:14.129917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 631/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:13 2022] GET /identity/v3/role_assignments?user.id=aff2305249a242639e1415094bf9df33&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=2302517080ac42578aa81c606dd0ff2f => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:16.233538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05738ba4-c2db-480a-8f9d-83174be84fb8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:16.234404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05738ba4-c2db-480a-8f9d-83174be84fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:16.234574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05738ba4-c2db-480a-8f9d-83174be84fb8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:16.236711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 630/1261] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:05:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:16.243190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118394b4-2400-4465-a532-334acb2bf07e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:16.243875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118394b4-2400-4465-a532-334acb2bf07e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:16.243989 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118394b4-2400-4465-a532-334acb2bf07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:16.265095 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-118394b4-2400-4465-a532-334acb2bf07e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:16.266313 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-118394b4-2400-4465-a532-334acb2bf07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:16.335147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 632/1262] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:05:16.357846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05738ba4-c2db-480a-8f9d-83174be84fb8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:16.372935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:16.475170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05738ba4-c2db-480a-8f9d-83174be84fb8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:16.523700 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c934f58-4fbf-43ed-a409-5f476059ad35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:16.524492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c934f58-4fbf-43ed-a409-5f476059ad35 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:16.524783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c934f58-4fbf-43ed-a409-5f476059ad35 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:16.525062 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c934f58-4fbf-43ed-a409-5f476059ad35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:16.538292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c934f58-4fbf-43ed-a409-5f476059ad35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2c934f58-4fbf-43ed-a409-5f476059ad35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:16.545751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:16.546877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:16.547975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:16.550073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 631/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:16.553325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118394b4-2400-4465-a532-334acb2bf07e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:16.572855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:16.660172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118394b4-2400-4465-a532-334acb2bf07e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:16.700917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07e6d1-2826-41c7-995d-7b49d5c9e8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:16.701577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07e6d1-2826-41c7-995d-7b49d5c9e8dd None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:16.701864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07e6d1-2826-41c7-995d-7b49d5c9e8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:16.702177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07e6d1-2826-41c7-995d-7b49d5c9e8dd None admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:16.702930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07e6d1-2826-41c7-995d-7b49d5c9e8dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:16.703240 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07e6d1-2826-41c7-995d-7b49d5c9e8dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:16.704613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07e6d1-2826-41c7-995d-7b49d5c9e8dd None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:16.713793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 633/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:16.720531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c934f58-4fbf-43ed-a409-5f476059ad35 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:16.798473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:16.838335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c934f58-4fbf-43ed-a409-5f476059ad35 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:16.873284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552516e8-fc4f-4fd5-a131-15fc0701d1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:16.874170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552516e8-fc4f-4fd5-a131-15fc0701d1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:16.874494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552516e8-fc4f-4fd5-a131-15fc0701d1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:16.874850 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552516e8-fc4f-4fd5-a131-15fc0701d1b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:16.887803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552516e8-fc4f-4fd5-a131-15fc0701d1b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:16.889400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552516e8-fc4f-4fd5-a131-15fc0701d1b3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-552516e8-fc4f-4fd5-a131-15fc0701d1b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:16.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:16.909063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:16.910217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:16.910217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:16.910217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:16.910217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:16.910217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 632/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:16.915140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07e6d1-2826-41c7-995d-7b49d5c9e8dd None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:17.003056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:17.023889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07e6d1-2826-41c7-995d-7b49d5c9e8dd None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:17.067166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7bd03-745d-4749-83b9-9e2fd10d29ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:17.067757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7bd03-745d-4749-83b9-9e2fd10d29ed None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:17.068044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7bd03-745d-4749-83b9-9e2fd10d29ed None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:17.068331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7bd03-745d-4749-83b9-9e2fd10d29ed None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:17.069059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7bd03-745d-4749-83b9-9e2fd10d29ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:17.069368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7bd03-745d-4749-83b9-9e2fd10d29ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:17.070769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7bd03-745d-4749-83b9-9e2fd10d29ed None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:17.082738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 634/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:17.088697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552516e8-fc4f-4fd5-a131-15fc0701d1b3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:17.199653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552516e8-fc4f-4fd5-a131-15fc0701d1b3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:17.225241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:17.246209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affce99c-fa36-45d6-8e51-83f4d69cbf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:17.247020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affce99c-fa36-45d6-8e51-83f4d69cbf9e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:17.247292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affce99c-fa36-45d6-8e51-83f4d69cbf9e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:17.247573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affce99c-fa36-45d6-8e51-83f4d69cbf9e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:17.255848 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affce99c-fa36-45d6-8e51-83f4d69cbf9e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:17.257620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affce99c-fa36-45d6-8e51-83f4d69cbf9e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-affce99c-fa36-45d6-8e51-83f4d69cbf9e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:05:17.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:17.273414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:17.274456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:17.274456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:17.274456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:05:17.274456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:05:17.274456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:05:17.274456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:05:17.274456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:05:17.274456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:05:17.274456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:05:17.274456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:17.274456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 633/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:17.279628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7bd03-745d-4749-83b9-9e2fd10d29ed None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:17.387991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7bd03-745d-4749-83b9-9e2fd10d29ed None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:17.423566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b39a0a-fc46-45ba-9011-bd9564e31f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:17.424223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b39a0a-fc46-45ba-9011-bd9564e31f24 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:17.424508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b39a0a-fc46-45ba-9011-bd9564e31f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:17.424778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b39a0a-fc46-45ba-9011-bd9564e31f24 None admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:17.425623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b39a0a-fc46-45ba-9011-bd9564e31f24 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:17.425928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b39a0a-fc46-45ba-9011-bd9564e31f24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:17.427560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b39a0a-fc46-45ba-9011-bd9564e31f24 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:17.436313 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:17.450455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 635/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 4 09:05:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:17.456316 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affce99c-fa36-45d6-8e51-83f4d69cbf9e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:17.576203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affce99c-fa36-45d6-8e51-83f4d69cbf9e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:17.610428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae050894-d914-4b29-af42-6debfb6f514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:17.611200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae050894-d914-4b29-af42-6debfb6f514d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:17.611479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae050894-d914-4b29-af42-6debfb6f514d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:17.611750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae050894-d914-4b29-af42-6debfb6f514d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:17.622615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae050894-d914-4b29-af42-6debfb6f514d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:17.624047 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae050894-d914-4b29-af42-6debfb6f514d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ae050894-d914-4b29-af42-6debfb6f514d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:17.641494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:17.642571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:17.643932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:17.643932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:17.643932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:17.643932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:17.643932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 634/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:17.649465 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b39a0a-fc46-45ba-9011-bd9564e31f24 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:17.752911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b39a0a-fc46-45ba-9011-bd9564e31f24 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:17.790338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677a6f8-3a86-4625-a128-f257a7c53fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:17.790986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677a6f8-3a86-4625-a128-f257a7c53fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:17.791269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677a6f8-3a86-4625-a128-f257a7c53fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:17.791545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677a6f8-3a86-4625-a128-f257a7c53fc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:17.792266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677a6f8-3a86-4625-a128-f257a7c53fc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:17.792578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677a6f8-3a86-4625-a128-f257a7c53fc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:17.793933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677a6f8-3a86-4625-a128-f257a7c53fc3 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:17.805559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 636/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:17.811610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae050894-d914-4b29-af42-6debfb6f514d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:17.849967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:17.928958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae050894-d914-4b29-af42-6debfb6f514d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:17.963699 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1c987-2199-4e3a-9e37-ad14b1838c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:17.964483 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1c987-2199-4e3a-9e37-ad14b1838c1e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:17.964759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1c987-2199-4e3a-9e37-ad14b1838c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:17.965031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1c987-2199-4e3a-9e37-ad14b1838c1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:17.972062 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1c987-2199-4e3a-9e37-ad14b1838c1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:17.973543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1c987-2199-4e3a-9e37-ad14b1838c1e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-7ef1c987-2199-4e3a-9e37-ad14b1838c1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:05:17.987905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:17.989041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:17.990023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:05:17.990023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:05:17.990023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:05:17.990023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:05:17.990023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:05:17.990023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:05:17.990023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:17.990023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:17.990023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 635/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:05:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:17.995468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677a6f8-3a86-4625-a128-f257a7c53fc3 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:18.061906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:18.107601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677a6f8-3a86-4625-a128-f257a7c53fc3 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:18.145760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4d414-5290-49d2-8a40-1b94e4d659f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:18.146455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4d414-5290-49d2-8a40-1b94e4d659f8 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:18.146777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4d414-5290-49d2-8a40-1b94e4d659f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:18.147052 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4d414-5290-49d2-8a40-1b94e4d659f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:18.147872 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4d414-5290-49d2-8a40-1b94e4d659f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:18.148186 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4d414-5290-49d2-8a40-1b94e4d659f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:18.149619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4d414-5290-49d2-8a40-1b94e4d659f8 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:18.162930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 637/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:05:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:18.169728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1c987-2199-4e3a-9e37-ad14b1838c1e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:18.279040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:18.289029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1c987-2199-4e3a-9e37-ad14b1838c1e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:18.323158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37116f9-eaba-487f-b9a1-0261848925b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:18.324042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37116f9-eaba-487f-b9a1-0261848925b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:18.324322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37116f9-eaba-487f-b9a1-0261848925b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:18.324598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37116f9-eaba-487f-b9a1-0261848925b0 None admin] PATH_INFO: `/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/aff2305249a242639e1415094bf9df33/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:18.326538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37116f9-eaba-487f-b9a1-0261848925b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c526301d4c7441396af366e54d9b3b5, user_id=aff2305249a242639e1415094bf9df33, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:18.327967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37116f9-eaba-487f-b9a1-0261848925b0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:18.346790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 636/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 4 09:05:18 2022] PUT /identity/v3/projects/4c526301d4c7441396af366e54d9b3b5/users/aff2305249a242639e1415094bf9df33/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:05:20.345660 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba06c43-952b-42e6-9c30-439d77b9cbe7 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:20.346612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba06c43-952b-42e6-9c30-439d77b9cbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:20.346612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba06c43-952b-42e6-9c30-439d77b9cbe7 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:20.348929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 638/1274] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:05:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:20.354616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056ce2-864c-4b05-8336-25e63224cf7b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:20.355056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056ce2-864c-4b05-8336-25e63224cf7b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:20.355146 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056ce2-864c-4b05-8336-25e63224cf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:20.373613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-20056ce2-864c-4b05-8336-25e63224cf7b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:20.374813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-20056ce2-864c-4b05-8336-25e63224cf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:20.450752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 637/1275] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:05:20.482943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba06c43-952b-42e6-9c30-439d77b9cbe7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:20.590509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba06c43-952b-42e6-9c30-439d77b9cbe7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:20.608530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:20.629919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93faa3cf-e0eb-4a75-b673-bc115e624c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:20.630722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faa3cf-e0eb-4a75-b673-bc115e624c37 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:20.631008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faa3cf-e0eb-4a75-b673-bc115e624c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:20.631285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faa3cf-e0eb-4a75-b673-bc115e624c37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:20.642487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93faa3cf-e0eb-4a75-b673-bc115e624c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-93faa3cf-e0eb-4a75-b673-bc115e624c37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 04 09:05:20.650195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:20.651275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:20.652494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 639/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 4 09:05:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:20.657651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20056ce2-864c-4b05-8336-25e63224cf7b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:20.738600 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:20.764470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20056ce2-864c-4b05-8336-25e63224cf7b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:20.799409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2911ea-a137-4f97-9e8d-ca403c037694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:20.800064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2911ea-a137-4f97-9e8d-ca403c037694 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:20.800347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2911ea-a137-4f97-9e8d-ca403c037694 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:20.800620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2911ea-a137-4f97-9e8d-ca403c037694 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:20.801377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2911ea-a137-4f97-9e8d-ca403c037694 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:20.801735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2911ea-a137-4f97-9e8d-ca403c037694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:20.803143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2911ea-a137-4f97-9e8d-ca403c037694 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:20.811252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 638/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 4 09:05:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:20.817615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93faa3cf-e0eb-4a75-b673-bc115e624c37 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:20.933853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93faa3cf-e0eb-4a75-b673-bc115e624c37 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:20.967152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d38f3-5049-4088-bf78-df9f912f12c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:20.967931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d38f3-5049-4088-bf78-df9f912f12c4 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:20.968211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d38f3-5049-4088-bf78-df9f912f12c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:20.968476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d38f3-5049-4088-bf78-df9f912f12c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:20.979354 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d38f3-5049-4088-bf78-df9f912f12c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:20.982230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d38f3-5049-4088-bf78-df9f912f12c4 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-b24d38f3-5049-4088-bf78-df9f912f12c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:20.999342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:21.000347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:21.001277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:21.001277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:21.001277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:21.001277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:21.001277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 640/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:21.006806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2911ea-a137-4f97-9e8d-ca403c037694 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:21.122223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2911ea-a137-4f97-9e8d-ca403c037694 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:21.161743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa723b2-063c-428c-816d-89c41dd505a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:21.162483 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa723b2-063c-428c-816d-89c41dd505a1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:21.162774 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa723b2-063c-428c-816d-89c41dd505a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:21.163057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa723b2-063c-428c-816d-89c41dd505a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:21.163792 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa723b2-063c-428c-816d-89c41dd505a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:21.164100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa723b2-063c-428c-816d-89c41dd505a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:21.165487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa723b2-063c-428c-816d-89c41dd505a1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:21.177463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 639/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:21.183837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d38f3-5049-4088-bf78-df9f912f12c4 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:21.219826 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:21.302076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d38f3-5049-4088-bf78-df9f912f12c4 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:21.336532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5780a7-a427-4786-8b51-f1665d725a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:21.337306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5780a7-a427-4786-8b51-f1665d725a8e None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:21.337584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5780a7-a427-4786-8b51-f1665d725a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:21.337849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5780a7-a427-4786-8b51-f1665d725a8e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:21.345309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5780a7-a427-4786-8b51-f1665d725a8e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:21.346784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5780a7-a427-4786-8b51-f1665d725a8e None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-aa5780a7-a427-4786-8b51-f1665d725a8e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 04 09:05:21.360982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:21.362247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:21.363891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:21.363891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:21.363891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 04 09:05:21.363891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 04 09:05:21.363891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 04 09:05:21.363891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 04 09:05:21.363891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 04 09:05:21.363891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 04 09:05:21.363891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 04 09:05:21.363891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:21.363891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 641/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:21.371058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa723b2-063c-428c-816d-89c41dd505a1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:21.377793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:21.474714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa723b2-063c-428c-816d-89c41dd505a1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:21.509195 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92da96-7ad1-4366-b9dd-4ff729e05a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:21.509850 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92da96-7ad1-4366-b9dd-4ff729e05a86 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:21.510160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92da96-7ad1-4366-b9dd-4ff729e05a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:21.510432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92da96-7ad1-4366-b9dd-4ff729e05a86 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:21.511255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92da96-7ad1-4366-b9dd-4ff729e05a86 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:21.511555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92da96-7ad1-4366-b9dd-4ff729e05a86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:21.512968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92da96-7ad1-4366-b9dd-4ff729e05a86 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:21.530061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 640/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 4 09:05:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:21.536036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5780a7-a427-4786-8b51-f1665d725a8e None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:21.632803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:21.643791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5780a7-a427-4786-8b51-f1665d725a8e None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:21.678399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe036f4-d2d0-4d0c-81e7-960995a98701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:21.679033 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe036f4-d2d0-4d0c-81e7-960995a98701 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:21.679311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe036f4-d2d0-4d0c-81e7-960995a98701 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:21.679586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe036f4-d2d0-4d0c-81e7-960995a98701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:21.747756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe036f4-d2d0-4d0c-81e7-960995a98701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:21.749196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe036f4-d2d0-4d0c-81e7-960995a98701 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:21.837868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:21.852196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 642/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 4 09:05:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 04 09:05:21.858489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92da96-7ad1-4366-b9dd-4ff729e05a86 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:21.931739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:21.971575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92da96-7ad1-4366-b9dd-4ff729e05a86 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.008036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75625b57-dcf5-4801-ac9a-11cc9e158449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.008827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75625b57-dcf5-4801-ac9a-11cc9e158449 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.009112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75625b57-dcf5-4801-ac9a-11cc9e158449 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.009387 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75625b57-dcf5-4801-ac9a-11cc9e158449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.019971 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75625b57-dcf5-4801-ac9a-11cc9e158449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:22.021460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75625b57-dcf5-4801-ac9a-11cc9e158449 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-75625b57-dcf5-4801-ac9a-11cc9e158449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 04 09:05:22.039182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:22.040158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:22.041121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 04 09:05:22.041121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 04 09:05:22.041121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 04 09:05:22.041121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:22.041121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 641/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 04 09:05:22.049363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe036f4-d2d0-4d0c-81e7-960995a98701 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.163733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe036f4-d2d0-4d0c-81e7-960995a98701 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.201237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e280da-16fb-4a42-a8fa-f773fe60c586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.201812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e280da-16fb-4a42-a8fa-f773fe60c586 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.202126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e280da-16fb-4a42-a8fa-f773fe60c586 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.202399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e280da-16fb-4a42-a8fa-f773fe60c586 None admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.203115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e280da-16fb-4a42-a8fa-f773fe60c586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:22.203413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e280da-16fb-4a42-a8fa-f773fe60c586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:22.204773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e280da-16fb-4a42-a8fa-f773fe60c586 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:22.219537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 643/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:22.226385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75625b57-dcf5-4801-ac9a-11cc9e158449 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.351884 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75625b57-dcf5-4801-ac9a-11cc9e158449 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.367096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:22.392170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d9ef4-9f6b-4b7d-9071-106f9c8c8be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.392972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d9ef4-9f6b-4b7d-9071-106f9c8c8be3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.393259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d9ef4-9f6b-4b7d-9071-106f9c8c8be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.393532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d9ef4-9f6b-4b7d-9071-106f9c8c8be3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.401342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d9ef4-9f6b-4b7d-9071-106f9c8c8be3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:22.402820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d9ef4-9f6b-4b7d-9071-106f9c8c8be3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2a9d9ef4-9f6b-4b7d-9071-106f9c8c8be3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 04 09:05:22.416972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:22.418018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:22.419157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 04 09:05:22.419157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 04 09:05:22.419157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 04 09:05:22.419157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 04 09:05:22.419157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 04 09:05:22.419157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 04 09:05:22.419157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 04 09:05:22.419157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 04 09:05:22.419157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 642/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:22.424235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e280da-16fb-4a42-a8fa-f773fe60c586 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.485164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:22.554522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e280da-16fb-4a42-a8fa-f773fe60c586 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.588747 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc17138-223f-4e9d-825c-9a5e0042ba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.589401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc17138-223f-4e9d-825c-9a5e0042ba13 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.589684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc17138-223f-4e9d-825c-9a5e0042ba13 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.589993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc17138-223f-4e9d-825c-9a5e0042ba13 None admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.590783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc17138-223f-4e9d-825c-9a5e0042ba13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:22.591094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc17138-223f-4e9d-825c-9a5e0042ba13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:22.592530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc17138-223f-4e9d-825c-9a5e0042ba13 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:22.604195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 644/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:22.610607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d9ef4-9f6b-4b7d-9071-106f9c8c8be3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:22.721417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d9ef4-9f6b-4b7d-9071-106f9c8c8be3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:22.756494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f8582-435c-440f-ae6f-3807ea3ebfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:22.757383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f8582-435c-440f-ae6f-3807ea3ebfea None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:22.757665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f8582-435c-440f-ae6f-3807ea3ebfea None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:22.757979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f8582-435c-440f-ae6f-3807ea3ebfea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:22.758801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f8582-435c-440f-ae6f-3807ea3ebfea None admin] RBAC: Adding query filter params (role.id=2302517080ac42578aa81c606dd0ff2f, scope.project.id=4c526301d4c7441396af366e54d9b3b5, user.id=aff2305249a242639e1415094bf9df33) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:22.759111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f8582-435c-440f-ae6f-3807ea3ebfea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:22.760552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f8582-435c-440f-ae6f-3807ea3ebfea None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:22.766575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 643/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 4 09:05:22 2022] GET /identity/v3/role_assignments?user.id=aff2305249a242639e1415094bf9df33&scope.project.id=4c526301d4c7441396af366e54d9b3b5&role.id=2302517080ac42578aa81c606dd0ff2f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:05:24.778022 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cfc6c-5501-42d0-8698-18285d63220b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:24.778592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cfc6c-5501-42d0-8698-18285d63220b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:24.778592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cfc6c-5501-42d0-8698-18285d63220b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:24.779937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 645/1288] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:24.785295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eff98-7836-40e6-8ebd-853c9287a6b8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:24.785755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eff98-7836-40e6-8ebd-853c9287a6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:24.785856 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eff98-7836-40e6-8ebd-853c9287a6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:24.804645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5e2eff98-7836-40e6-8ebd-853c9287a6b8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:24.805823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5e2eff98-7836-40e6-8ebd-853c9287a6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:24.835652 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:24.875365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 644/1289] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:05:24.904332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cfc6c-5501-42d0-8698-18285d63220b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:25.019080 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cfc6c-5501-42d0-8698-18285d63220b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:25.031399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:25.060188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf0deb-8ac9-469a-a4a8-52069a55f6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:25.060982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf0deb-8ac9-469a-a4a8-52069a55f6cf None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.061223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf0deb-8ac9-469a-a4a8-52069a55f6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.061526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf0deb-8ac9-469a-a4a8-52069a55f6cf None admin] PATH_INFO: `/v3/services/placement` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.062209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf0deb-8ac9-469a-a4a8-52069a55f6cf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:25.063577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf0deb-8ac9-469a-a4a8-52069a55f6cf None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-4ddf0deb-8ac9-469a-a4a8-52069a55f6cf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:25.076182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:25.077331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:25.078255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 646/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 4 09:05:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:25.085746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eff98-7836-40e6-8ebd-853c9287a6b8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:25.191165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eff98-7836-40e6-8ebd-853c9287a6b8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:25.226894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e8a4f-e956-4380-a273-e30073bbdb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:25.227549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e8a4f-e956-4380-a273-e30073bbdb28 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.227829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e8a4f-e956-4380-a273-e30073bbdb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.228100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e8a4f-e956-4380-a273-e30073bbdb28 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.228808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e8a4f-e956-4380-a273-e30073bbdb28 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:25.229107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e8a4f-e956-4380-a273-e30073bbdb28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:25.230656 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e8a4f-e956-4380-a273-e30073bbdb28 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:25.235628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 645/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 4 09:05:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:25.244379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf0deb-8ac9-469a-a4a8-52069a55f6cf None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:25.356535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf0deb-8ac9-469a-a4a8-52069a55f6cf None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:25.393853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765803e9-1a97-488a-9b75-319b8a7a7970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:25.394616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765803e9-1a97-488a-9b75-319b8a7a7970 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:25.394897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765803e9-1a97-488a-9b75-319b8a7a7970 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:25.395224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765803e9-1a97-488a-9b75-319b8a7a7970 None admin] PATH_INFO: `/v3/services` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:25.395945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765803e9-1a97-488a-9b75-319b8a7a7970 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:25.396251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765803e9-1a97-488a-9b75-319b8a7a7970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:25.397625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765803e9-1a97-488a-9b75-319b8a7a7970 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:25.402655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 647/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 4 09:05:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:27.383669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96b2fc-c0c1-4031-8ce7-085924bee006 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:27.384269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96b2fc-c0c1-4031-8ce7-085924bee006 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:27.384269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96b2fc-c0c1-4031-8ce7-085924bee006 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:27.385608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 646/1293] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:27.391889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68caf9-e6f3-412a-96b2-6aa99bdea46e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:27.392626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68caf9-e6f3-412a-96b2-6aa99bdea46e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:27.392768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68caf9-e6f3-412a-96b2-6aa99bdea46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:27.418401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8c68caf9-e6f3-412a-96b2-6aa99bdea46e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:27.419514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8c68caf9-e6f3-412a-96b2-6aa99bdea46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:27.454039 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:27.493957 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 648/1294] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:05:27.521331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96b2fc-c0c1-4031-8ce7-085924bee006 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:27.557133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:27.631144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96b2fc-c0c1-4031-8ce7-085924bee006 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:27.666817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89bb5c-410e-4ed7-80f8-76eb6260338c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:27.667480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89bb5c-410e-4ed7-80f8-76eb6260338c None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:27.667763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89bb5c-410e-4ed7-80f8-76eb6260338c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:27.668042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89bb5c-410e-4ed7-80f8-76eb6260338c None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:27.668829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89bb5c-410e-4ed7-80f8-76eb6260338c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:27.670164 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89bb5c-410e-4ed7-80f8-76eb6260338c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:27.686233 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 647/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 4 09:05:27 2022] POST /identity/v3/services => generated 246 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:05:29.699161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6ea95-acab-4d70-9d46-d028ce253f48 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:29.700093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6ea95-acab-4d70-9d46-d028ce253f48 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:29.700093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6ea95-acab-4d70-9d46-d028ce253f48 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:29.702297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 649/1296] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:05:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:29.708954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b029d10-6a3b-4c77-a289-f536c720ccf2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:29.709669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b029d10-6a3b-4c77-a289-f536c720ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:29.709864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b029d10-6a3b-4c77-a289-f536c720ccf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:29.733069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7b029d10-6a3b-4c77-a289-f536c720ccf2 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:29.734288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7b029d10-6a3b-4c77-a289-f536c720ccf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:29.820246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 648/1297] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:05:29.843240 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6ea95-acab-4d70-9d46-d028ce253f48 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:29.968519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6ea95-acab-4d70-9d46-d028ce253f48 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:30.002798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd957f7-39b7-4929-af0c-afc2825c15fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:30.003578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd957f7-39b7-4929-af0c-afc2825c15fb None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:30.003858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd957f7-39b7-4929-af0c-afc2825c15fb None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:30.004134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd957f7-39b7-4929-af0c-afc2825c15fb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:30.004806 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd957f7-39b7-4929-af0c-afc2825c15fb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:30.006229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd957f7-39b7-4929-af0c-afc2825c15fb None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-5fd957f7-39b7-4929-af0c-afc2825c15fb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:30.018123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:30.019361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:30.020822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 650/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 4 09:05:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:30.025617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b029d10-6a3b-4c77-a289-f536c720ccf2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:30.047427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:30.135721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b029d10-6a3b-4c77-a289-f536c720ccf2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:30.172706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39620a-0e20-418b-8928-85540444bae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:30.173378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39620a-0e20-418b-8928-85540444bae4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:30.173660 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39620a-0e20-418b-8928-85540444bae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:30.173968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39620a-0e20-418b-8928-85540444bae4 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:30.174692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39620a-0e20-418b-8928-85540444bae4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:30.175000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39620a-0e20-418b-8928-85540444bae4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:30.176375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39620a-0e20-418b-8928-85540444bae4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:30.181486 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 649/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 4 09:05:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:30.188460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd957f7-39b7-4929-af0c-afc2825c15fb None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:30.225848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:30.302322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd957f7-39b7-4929-af0c-afc2825c15fb None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:30.338308 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97ab97-2d30-4d02-b6f0-88128e2a2aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:30.338952 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ab97-2d30-4d02-b6f0-88128e2a2aef None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:30.339237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ab97-2d30-4d02-b6f0-88128e2a2aef None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:30.339520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97ab97-2d30-4d02-b6f0-88128e2a2aef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:30.340312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97ab97-2d30-4d02-b6f0-88128e2a2aef None admin] RBAC: Adding query filter params (interface=public, service_id=42704510df234db8a0a3929e59086195, region_id=RegionOne) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:30.340633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97ab97-2d30-4d02-b6f0-88128e2a2aef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:30.342027 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97ab97-2d30-4d02-b6f0-88128e2a2aef None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:30.347255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 651/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 4 09:05:30 2022] GET /identity/v3/endpoints?service_id=42704510df234db8a0a3929e59086195&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:30.356350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39620a-0e20-418b-8928-85540444bae4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:30.451578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:30.471616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39620a-0e20-418b-8928-85540444bae4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:30.510207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f4120-d113-4058-8c65-273f3bdb4426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:30.510803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f4120-d113-4058-8c65-273f3bdb4426 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:30.511090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f4120-d113-4058-8c65-273f3bdb4426 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:30.511369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f4120-d113-4058-8c65-273f3bdb4426 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:30.512059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f4120-d113-4058-8c65-273f3bdb4426 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:30.513431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f4120-d113-4058-8c65-273f3bdb4426 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:30.525776 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 650/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 4 09:05:30 2022] GET /identity/v3/services => generated 2191 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:05:32.500431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d640dc-e977-499e-b47e-1dc276ca09ad None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:32.500994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d640dc-e977-499e-b47e-1dc276ca09ad None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:32.500994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d640dc-e977-499e-b47e-1dc276ca09ad None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:32.502290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 652/1302] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:05:32.508920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46e9d-5d55-4440-8fc5-85e81acbe034 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:32.509637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46e9d-5d55-4440-8fc5-85e81acbe034 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:32.509988 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46e9d-5d55-4440-8fc5-85e81acbe034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:32.532500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0ce46e9d-5d55-4440-8fc5-85e81acbe034 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:05:32.533696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0ce46e9d-5d55-4440-8fc5-85e81acbe034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:05:32.610958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 651/1303] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:05:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 04 09:05:32.635548 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d640dc-e977-499e-b47e-1dc276ca09ad None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:32.741744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:32.749773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d640dc-e977-499e-b47e-1dc276ca09ad None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:32.783356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3cd04-e024-4404-b2d8-3c79a14956d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:32.784090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3cd04-e024-4404-b2d8-3c79a14956d5 None admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:32.784374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3cd04-e024-4404-b2d8-3c79a14956d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:32.784651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3cd04-e024-4404-b2d8-3c79a14956d5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:32.785304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3cd04-e024-4404-b2d8-3c79a14956d5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:32.786728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3cd04-e024-4404-b2d8-3c79a14956d5 None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application [None req-1ae3cd04-e024-4404-b2d8-3c79a14956d5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 04 09:05:32.798300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 04 09:05:32.799397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.get_or_create( Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application with Lock( Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self._enter() Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application return self.creator() Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application created_value = creator( Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: ERROR keystone.server.flask.application Jul 04 09:05:32.800274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 653/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 4 09:05:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 04 09:05:32.806053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce46e9d-5d55-4440-8fc5-85e81acbe034 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:32.886998 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:32.911280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce46e9d-5d55-4440-8fc5-85e81acbe034 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:32.947544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133a2d-b491-42f8-9843-516d6cc0f382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:32.948225 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133a2d-b491-42f8-9843-516d6cc0f382 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:32.948512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133a2d-b491-42f8-9843-516d6cc0f382 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:32.948822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133a2d-b491-42f8-9843-516d6cc0f382 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:32.949538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd133a2d-b491-42f8-9843-516d6cc0f382 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:05:32.949846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd133a2d-b491-42f8-9843-516d6cc0f382 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:32.951263 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd133a2d-b491-42f8-9843-516d6cc0f382 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:32.956589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 652/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 4 09:05:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:05:32.962761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3cd04-e024-4404-b2d8-3c79a14956d5 None admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:05:33.086797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3cd04-e024-4404-b2d8-3c79a14956d5 None admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:05:33.131784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28801103-c281-416e-abb0-a0dd9269e70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:05:33.132860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28801103-c281-416e-abb0-a0dd9269e70b None admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:05:33.133143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28801103-c281-416e-abb0-a0dd9269e70b None admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:05:33.133420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28801103-c281-416e-abb0-a0dd9269e70b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:05:33.134200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28801103-c281-416e-abb0-a0dd9269e70b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:05:33.135575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28801103-c281-416e-abb0-a0dd9269e70b None admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:05:33.157071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 654/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 4 09:05:32 2022] POST /identity/v3/endpoints => generated 331 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:05:42.953525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=30 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:05:43.152601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:36.245911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c889117-a12a-463c-a2ea-73dba3d3e699 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.246409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c889117-a12a-463c-a2ea-73dba3d3e699 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.246501 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c889117-a12a-463c-a2ea-73dba3d3e699 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.247854 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 653/1307] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:36.253205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccaa5e-edd9-4984-a3a3-fc4e315db17a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.253593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccaa5e-edd9-4984-a3a3-fc4e315db17a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.253833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccaa5e-edd9-4984-a3a3-fc4e315db17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.338395 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e4ccaa5e-edd9-4984-a3a3-fc4e315db17a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:36.339630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e4ccaa5e-edd9-4984-a3a3-fc4e315db17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:36.470240 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 655/1308] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:06:36.517389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc52fa-6e37-4f06-875a-fabf80e51568 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.518121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc52fa-6e37-4f06-875a-fabf80e51568 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.518870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc52fa-6e37-4f06-875a-fabf80e51568 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.521968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 654/1309] 10.222.0.26 () {54 vars in 964 bytes} [Mon Jul 4 09:06:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:36.540631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcd271-d3fb-4265-8dea-eaae644f4cd4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.541489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcd271-d3fb-4265-8dea-eaae644f4cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.542003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcd271-d3fb-4265-8dea-eaae644f4cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.628477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e3dcd271-d3fb-4265-8dea-eaae644f4cd4 None None] MFA Rules not processed for user `98649aa38f764af0b41ff5594b94a989`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:36.629752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e3dcd271-d3fb-4265-8dea-eaae644f4cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:36.653598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:36.718005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 656/1310] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Jul 4 09:06:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:06:36.727233 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc52fa-6e37-4f06-875a-fabf80e51568 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:36.729163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1ffc52fa-6e37-4f06-875a-fabf80e51568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:36.842931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc52fa-6e37-4f06-875a-fabf80e51568 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:36.921465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaddcb5-7401-432d-bc43-a1159ccacc0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:36.922168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaddcb5-7401-432d-bc43-a1159ccacc0b service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:36.922454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaddcb5-7401-432d-bc43-a1159ccacc0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:36.922730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaddcb5-7401-432d-bc43-a1159ccacc0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:36.924378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-caaddcb5-7401-432d-bc43-a1159ccacc0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:36.982117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:37.041565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaddcb5-7401-432d-bc43-a1159ccacc0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:37.043096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaddcb5-7401-432d-bc43-a1159ccacc0b service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:37.062179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 655/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:06:39.353225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28710-6941-4538-8634-b9619b9e1106 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.353785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28710-6941-4538-8634-b9619b9e1106 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.353785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28710-6941-4538-8634-b9619b9e1106 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.355350 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 657/1312] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:39.360930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2cd3b-5256-406a-8c3e-7f60163d7b14 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.361539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2cd3b-5256-406a-8c3e-7f60163d7b14 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.361539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2cd3b-5256-406a-8c3e-7f60163d7b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.439354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-75e2cd3b-5256-406a-8c3e-7f60163d7b14 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:39.440476 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-75e2cd3b-5256-406a-8c3e-7f60163d7b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:39.554453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 656/1313] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:06:39.608994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404cf90-f7cc-411e-ab4d-fe4269efda89 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.609278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404cf90-f7cc-411e-ab4d-fe4269efda89 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.609551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404cf90-f7cc-411e-ab4d-fe4269efda89 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.610976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 658/1314] 10.222.0.26 () {54 vars in 964 bytes} [Mon Jul 4 09:06:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:39.617008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2044d-4194-4f6f-b290-506fb6cf250f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:39.617298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2044d-4194-4f6f-b290-506fb6cf250f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:39.617575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2044d-4194-4f6f-b290-506fb6cf250f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:39.628328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:39.698751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-16a2044d-4194-4f6f-b290-506fb6cf250f None None] MFA Rules not processed for user `98649aa38f764af0b41ff5594b94a989`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:39.699890 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-16a2044d-4194-4f6f-b290-506fb6cf250f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:39.780411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 657/1315] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Jul 4 09:06:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:06:39.789657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404cf90-f7cc-411e-ab4d-fe4269efda89 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:39.791284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e404cf90-f7cc-411e-ab4d-fe4269efda89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:39.922043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404cf90-f7cc-411e-ab4d-fe4269efda89 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:39.996762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:39.999278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257568f2-41af-4b54-968f-2b250ed0357d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:39.999905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257568f2-41af-4b54-968f-2b250ed0357d service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:40.000134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257568f2-41af-4b54-968f-2b250ed0357d service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:40.000444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257568f2-41af-4b54-968f-2b250ed0357d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:40.002972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-257568f2-41af-4b54-968f-2b250ed0357d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:40.118945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257568f2-41af-4b54-968f-2b250ed0357d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:40.120396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257568f2-41af-4b54-968f-2b250ed0357d service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:40.140516 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 659/1316] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:06:42.688078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52ed9b-83bc-441d-bec1-6f9dd5444d8e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.688927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52ed9b-83bc-441d-bec1-6f9dd5444d8e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.688927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52ed9b-83bc-441d-bec1-6f9dd5444d8e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.691324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 658/1317] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:06:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:42.697444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48111591-3d23-4e88-ba4c-797bb1db10b9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:42.697926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48111591-3d23-4e88-ba4c-797bb1db10b9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:42.698024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48111591-3d23-4e88-ba4c-797bb1db10b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:42.769211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-48111591-3d23-4e88-ba4c-797bb1db10b9 None None] MFA Rules not processed for user `264b2fc6044940aba215d38e72c6e93a`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:42.770393 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-48111591-3d23-4e88-ba4c-797bb1db10b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:42.823597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:42.873397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 660/1318] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:06:42.916207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52ed9b-83bc-441d-bec1-6f9dd5444d8e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:42.918910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7b52ed9b-83bc-441d-bec1-6f9dd5444d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:43.035897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52ed9b-83bc-441d-bec1-6f9dd5444d8e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:43.038925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:43.118817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c846b-c01e-4b40-81f7-d36189d54773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:43.119812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c846b-c01e-4b40-81f7-d36189d54773 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:43.120167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c846b-c01e-4b40-81f7-d36189d54773 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:43.120523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c846b-c01e-4b40-81f7-d36189d54773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:43.123248 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-056c846b-c01e-4b40-81f7-d36189d54773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:43.240836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c846b-c01e-4b40-81f7-d36189d54773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:43.242332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c846b-c01e-4b40-81f7-d36189d54773 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:43.262239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 659/1319] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:06:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:06:45.986866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eddcd0-847d-4f4b-8d20-1d6e61b536cb None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.987523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eddcd0-847d-4f4b-8d20-1d6e61b536cb None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.987523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eddcd0-847d-4f4b-8d20-1d6e61b536cb None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:45.989657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 661/1320] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:06:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:45.995904 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd81ddd-84d5-48e5-ba90-1877cc341d13 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:45.996651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd81ddd-84d5-48e5-ba90-1877cc341d13 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:45.996828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd81ddd-84d5-48e5-ba90-1877cc341d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.044135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:46.073911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0cd81ddd-84d5-48e5-ba90-1877cc341d13 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:46.075116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0cd81ddd-84d5-48e5-ba90-1877cc341d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:46.180610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 660/1321] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:06:46.222212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eddcd0-847d-4f4b-8d20-1d6e61b536cb None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:46.223771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-75eddcd0-847d-4f4b-8d20-1d6e61b536cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:46.335996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eddcd0-847d-4f4b-8d20-1d6e61b536cb None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:46.411122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7807b86-fe14-4f8b-bb9b-fa44bf902f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:46.411764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7807b86-fe14-4f8b-bb9b-fa44bf902f5d service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:46.412189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7807b86-fe14-4f8b-bb9b-fa44bf902f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:46.412324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7807b86-fe14-4f8b-bb9b-fa44bf902f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:46.414051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d7807b86-fe14-4f8b-bb9b-fa44bf902f5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:46.421630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:46.537088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7807b86-fe14-4f8b-bb9b-fa44bf902f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:46.538691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7807b86-fe14-4f8b-bb9b-fa44bf902f5d service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:46.558995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 662/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:06:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:06:49.290534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b2ec6-ce70-4f82-b611-88786bc11a87 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.291108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b2ec6-ce70-4f82-b611-88786bc11a87 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.291108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b2ec6-ce70-4f82-b611-88786bc11a87 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.292464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 661/1323] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:49.297918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ba6ca-1db1-4a95-a2c5-99ae657e27eb None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.298375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ba6ca-1db1-4a95-a2c5-99ae657e27eb None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.298469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ba6ca-1db1-4a95-a2c5-99ae657e27eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.366178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-391ba6ca-1db1-4a95-a2c5-99ae657e27eb None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:49.367371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-391ba6ca-1db1-4a95-a2c5-99ae657e27eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:49.432264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:49.477290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 663/1324] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:06:49.514098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b2ec6-ce70-4f82-b611-88786bc11a87 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:49.515664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d06b2ec6-ce70-4f82-b611-88786bc11a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:49.640310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b2ec6-ce70-4f82-b611-88786bc11a87 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:49.653660 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:49.721594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2a1b5-5e5a-4626-90f0-1d514e516a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:49.722202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2a1b5-5e5a-4626-90f0-1d514e516a83 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:49.722494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2a1b5-5e5a-4626-90f0-1d514e516a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:49.722767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2a1b5-5e5a-4626-90f0-1d514e516a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:49.724458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c9a2a1b5-5e5a-4626-90f0-1d514e516a83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:49.840812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2a1b5-5e5a-4626-90f0-1d514e516a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:49.842308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2a1b5-5e5a-4626-90f0-1d514e516a83 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:49.861699 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 662/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:06:51.935908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70713a3-a99a-4d08-98ce-7afebf475af2 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.936447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70713a3-a99a-4d08-98ce-7afebf475af2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.936447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70713a3-a99a-4d08-98ce-7afebf475af2 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.937726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 664/1326] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:51.943654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5864e-b088-45eb-81b8-959cb9e65da1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:51.944107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5864e-b088-45eb-81b8-959cb9e65da1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:51.944204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5864e-b088-45eb-81b8-959cb9e65da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:51.992820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:52.013871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5cc5864e-b088-45eb-81b8-959cb9e65da1 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:52.015123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5cc5864e-b088-45eb-81b8-959cb9e65da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:52.123835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 663/1327] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:06:52.159725 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70713a3-a99a-4d08-98ce-7afebf475af2 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:52.161281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a70713a3-a99a-4d08-98ce-7afebf475af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:52.272051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70713a3-a99a-4d08-98ce-7afebf475af2 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:52.352924 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af140705-6595-426b-9d71-e681d19216e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:52.353576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af140705-6595-426b-9d71-e681d19216e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:52.353860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af140705-6595-426b-9d71-e681d19216e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:52.354164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af140705-6595-426b-9d71-e681d19216e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:52.355824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-af140705-6595-426b-9d71-e681d19216e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:52.373545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:52.478511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af140705-6595-426b-9d71-e681d19216e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:52.479991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af140705-6595-426b-9d71-e681d19216e5 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:52.499849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 665/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:06:54.563736 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673891df-3be7-4375-9872-b123f31501bd None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:54.564234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673891df-3be7-4375-9872-b123f31501bd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:54.564234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673891df-3be7-4375-9872-b123f31501bd None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:54.565631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 664/1329] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:54.571406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da5ad8-9546-47b9-96c9-2108f0faa987 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:54.571813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da5ad8-9546-47b9-96c9-2108f0faa987 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:54.571929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da5ad8-9546-47b9-96c9-2108f0faa987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:54.641502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-47da5ad8-9546-47b9-96c9-2108f0faa987 None None] MFA Rules not processed for user `264b2fc6044940aba215d38e72c6e93a`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:54.642759 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-47da5ad8-9546-47b9-96c9-2108f0faa987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:54.713358 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:54.747795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 666/1330] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:06:54.785932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673891df-3be7-4375-9872-b123f31501bd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:54.787712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-673891df-3be7-4375-9872-b123f31501bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:54.898707 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673891df-3be7-4375-9872-b123f31501bd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:54.920060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:54.975818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349c795-f8a7-42e8-ab3a-a0e26b210a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:54.976470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349c795-f8a7-42e8-ab3a-a0e26b210a0a service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:54.976752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349c795-f8a7-42e8-ab3a-a0e26b210a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:54.977029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349c795-f8a7-42e8-ab3a-a0e26b210a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:54.978721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9349c795-f8a7-42e8-ab3a-a0e26b210a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:55.114799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349c795-f8a7-42e8-ab3a-a0e26b210a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:55.116345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349c795-f8a7-42e8-ab3a-a0e26b210a0a service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:55.136704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 665/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:06:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:06:57.481565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f311df8-3df7-43fd-9b6c-e87e765a4254 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:57.482153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f311df8-3df7-43fd-9b6c-e87e765a4254 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:57.482153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f311df8-3df7-43fd-9b6c-e87e765a4254 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:57.483426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 667/1332] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:06:57.489963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441d84d-7d66-472e-9dfc-22b627835147 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:57.490438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441d84d-7d66-472e-9dfc-22b627835147 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:57.490528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441d84d-7d66-472e-9dfc-22b627835147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:57.569477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:57.572835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7441d84d-7d66-472e-9dfc-22b627835147 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:06:57.574976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7441d84d-7d66-472e-9dfc-22b627835147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:57.683843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 666/1333] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:06:57.722992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f311df8-3df7-43fd-9b6c-e87e765a4254 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:06:57.725423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2f311df8-3df7-43fd-9b6c-e87e765a4254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:57.845309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f311df8-3df7-43fd-9b6c-e87e765a4254 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:06:57.923726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786af87a-081e-468d-a279-f9a1200d71ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:06:57.924380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786af87a-081e-468d-a279-f9a1200d71ec service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:06:57.924669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786af87a-081e-468d-a279-f9a1200d71ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:06:57.924947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786af87a-081e-468d-a279-f9a1200d71ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:06:57.926664 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-786af87a-081e-468d-a279-f9a1200d71ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:06:57.956057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:06:58.047915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786af87a-081e-468d-a279-f9a1200d71ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:06:58.049277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786af87a-081e-468d-a279-f9a1200d71ec service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:06:58.067651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 668/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:00.151054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f1e4-5b87-4d27-af43-6a7f4c760178 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:00.151648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f1e4-5b87-4d27-af43-6a7f4c760178 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:00.151648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f1e4-5b87-4d27-af43-6a7f4c760178 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:00.153097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 667/1335] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:00.159520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272f0e1-6bd3-4799-b657-8a8c3df263f6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:00.160249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272f0e1-6bd3-4799-b657-8a8c3df263f6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:00.160344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272f0e1-6bd3-4799-b657-8a8c3df263f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:00.231910 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7272f0e1-6bd3-4799-b657-8a8c3df263f6 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:00.233110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7272f0e1-6bd3-4799-b657-8a8c3df263f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:00.318107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:00.342630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 669/1336] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:00.382960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327f1e4-5b87-4d27-af43-6a7f4c760178 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:00.385360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c327f1e4-5b87-4d27-af43-6a7f4c760178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:00.527499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327f1e4-5b87-4d27-af43-6a7f4c760178 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:00.556638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:00.607554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98760e-5eae-4952-aba3-312ce500da68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:00.608204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98760e-5eae-4952-aba3-312ce500da68 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:00.608492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98760e-5eae-4952-aba3-312ce500da68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:00.608768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98760e-5eae-4952-aba3-312ce500da68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:00.610459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0b98760e-5eae-4952-aba3-312ce500da68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:00.723446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98760e-5eae-4952-aba3-312ce500da68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:00.725022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98760e-5eae-4952-aba3-312ce500da68 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:00.745178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 668/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:03.098952 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bfc9f-5716-4063-a4cd-fbe0f46df7a7 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.099852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bfc9f-5716-4063-a4cd-fbe0f46df7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.099852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bfc9f-5716-4063-a4cd-fbe0f46df7a7 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.102063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 670/1338] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:03.110137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce412a-8634-44a2-a944-952d4a45397c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.110604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce412a-8634-44a2-a944-952d4a45397c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.110706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce412a-8634-44a2-a944-952d4a45397c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.179730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-58ce412a-8634-44a2-a944-952d4a45397c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:03.180956 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-58ce412a-8634-44a2-a944-952d4a45397c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:03.189578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:03.282929 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 669/1339] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:03.322705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bfc9f-5716-4063-a4cd-fbe0f46df7a7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:03.325066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f56bfc9f-5716-4063-a4cd-fbe0f46df7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:03.446869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bfc9f-5716-4063-a4cd-fbe0f46df7a7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:03.521933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88baf245-0d94-44e7-be03-0d8349c9e7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:03.522697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baf245-0d94-44e7-be03-0d8349c9e7ad service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:03.522985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baf245-0d94-44e7-be03-0d8349c9e7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:03.523263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baf245-0d94-44e7-be03-0d8349c9e7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:03.524910 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-88baf245-0d94-44e7-be03-0d8349c9e7ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:03.562035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:03.637336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88baf245-0d94-44e7-be03-0d8349c9e7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:03.638818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88baf245-0d94-44e7-be03-0d8349c9e7ad service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:03.657276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 671/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:07.871902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11c365-9605-4c31-adb2-787301131678 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:07.872449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11c365-9605-4c31-adb2-787301131678 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:07.872449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11c365-9605-4c31-adb2-787301131678 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:07.873795 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 670/1341] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:07.879098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced60d3f-30f9-4b51-88a9-989e55359185 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:07.879510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced60d3f-30f9-4b51-88a9-989e55359185 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:07.879733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced60d3f-30f9-4b51-88a9-989e55359185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:07.957617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ced60d3f-30f9-4b51-88a9-989e55359185 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:07.958866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ced60d3f-30f9-4b51-88a9-989e55359185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:08.058462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:08.072518 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 672/1342] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:08.111752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11c365-9605-4c31-adb2-787301131678 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:08.114124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3b11c365-9605-4c31-adb2-787301131678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:08.223646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11c365-9605-4c31-adb2-787301131678 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:08.227643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21c233-c352-4915-94f0-418cd7e92ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:08.228296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21c233-c352-4915-94f0-418cd7e92ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:08.228582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21c233-c352-4915-94f0-418cd7e92ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:08.228855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21c233-c352-4915-94f0-418cd7e92ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:08.238159 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21c233-c352-4915-94f0-418cd7e92ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:08.239641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21c233-c352-4915-94f0-418cd7e92ad6 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:08.249403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 671/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:10.869191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7a266-2f0b-4c7a-8119-fc41af25e827 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:10.869772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7a266-2f0b-4c7a-8119-fc41af25e827 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:10.869772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7a266-2f0b-4c7a-8119-fc41af25e827 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:10.871621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 673/1344] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:10.876786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dacb3-fea8-4303-bd3f-2fe58ebe4130 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:10.877285 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dacb3-fea8-4303-bd3f-2fe58ebe4130 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:10.877380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dacb3-fea8-4303-bd3f-2fe58ebe4130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:10.906610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3d7dacb3-fea8-4303-bd3f-2fe58ebe4130 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:10.907801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3d7dacb3-fea8-4303-bd3f-2fe58ebe4130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:10.940387 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:11.001500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 672/1345] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:11.038816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7a266-2f0b-4c7a-8119-fc41af25e827 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:11.040794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c2c7a266-2f0b-4c7a-8119-fc41af25e827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:11.055078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7a266-2f0b-4c7a-8119-fc41af25e827 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:11.058936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd3faa-4659-49a9-bcfb-3a6f9133eb63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:11.059605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd3faa-4659-49a9-bcfb-3a6f9133eb63 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:11.059894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd3faa-4659-49a9-bcfb-3a6f9133eb63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:11.060168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd3faa-4659-49a9-bcfb-3a6f9133eb63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:11.070356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd3faa-4659-49a9-bcfb-3a6f9133eb63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:11.071835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd3faa-4659-49a9-bcfb-3a6f9133eb63 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:11.082495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 674/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:15.320254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099be005-e89a-4563-b661-2157d968be42 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:15.320836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099be005-e89a-4563-b661-2157d968be42 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:15.320836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099be005-e89a-4563-b661-2157d968be42 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:15.322619 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 673/1347] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:15.328036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab9b05-0db4-4c48-9f6f-2d205c37575b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:15.328499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab9b05-0db4-4c48-9f6f-2d205c37575b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:15.328608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab9b05-0db4-4c48-9f6f-2d205c37575b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:15.347341 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-39ab9b05-0db4-4c48-9f6f-2d205c37575b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:15.348493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-39ab9b05-0db4-4c48-9f6f-2d205c37575b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:15.357116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 675/1348] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:15.395938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099be005-e89a-4563-b661-2157d968be42 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:15.408841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099be005-e89a-4563-b661-2157d968be42 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:15.413152 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc967298-25da-4420-9ad0-01ba033a70ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:15.413788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc967298-25da-4420-9ad0-01ba033a70ed service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:15.414104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc967298-25da-4420-9ad0-01ba033a70ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:15.414349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc967298-25da-4420-9ad0-01ba033a70ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:15.424086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc967298-25da-4420-9ad0-01ba033a70ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:15.425540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc967298-25da-4420-9ad0-01ba033a70ed service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:15.435726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 674/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:20.552096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc25b-c0be-4943-9c4b-71d1ffc6da56 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.562340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc25b-c0be-4943-9c4b-71d1ffc6da56 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.562956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc25b-c0be-4943-9c4b-71d1ffc6da56 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.565998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 676/1350] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:07:20 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:20.583948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dc3b6-1854-4ce5-b160-15822b1f0c75 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:20.585026 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dc3b6-1854-4ce5-b160-15822b1f0c75 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:20.585502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dc3b6-1854-4ce5-b160-15822b1f0c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:20.666191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a15dc3b6-1854-4ce5-b160-15822b1f0c75 None None] MFA Rules not processed for user `aff2305249a242639e1415094bf9df33`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:20.667303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a15dc3b6-1854-4ce5-b160-15822b1f0c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:20.742400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 675/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:07:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:24.746987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98c76e-4fee-474e-90e7-d2126aa5b6cf None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:24.747973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98c76e-4fee-474e-90e7-d2126aa5b6cf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:24.748289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98c76e-4fee-474e-90e7-d2126aa5b6cf None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:24.749930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 677/1352] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:24.755241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e7fb-3cd4-435f-bc88-e0bbaceac85e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:24.755669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e7fb-3cd4-435f-bc88-e0bbaceac85e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:24.755806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e7fb-3cd4-435f-bc88-e0bbaceac85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:24.775737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b381e7fb-3cd4-435f-bc88-e0bbaceac85e None None] MFA Rules not processed for user `aff2305249a242639e1415094bf9df33`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:24.776984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b381e7fb-3cd4-435f-bc88-e0bbaceac85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:24.786712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 676/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:07:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:25.342468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=13 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:26.425010 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbfdae-b0ab-4a03-b0fb-536435365f8f None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:26.425683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbfdae-b0ab-4a03-b0fb-536435365f8f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:26.425683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbfdae-b0ab-4a03-b0fb-536435365f8f None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:26.426858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 678/1354] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:26.432179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf826b-d391-4f21-b750-4895b4f058ca None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:26.432642 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf826b-d391-4f21-b750-4895b4f058ca None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:26.432979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf826b-d391-4f21-b750-4895b4f058ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:26.451842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-42cf826b-d391-4f21-b750-4895b4f058ca None None] MFA Rules not processed for user `aff2305249a242639e1415094bf9df33`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:26.453038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-42cf826b-d391-4f21-b750-4895b4f058ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:26.461800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 677/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:07:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:07:30.090845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f331fe-7c1c-4948-8267-0bc2f1938529 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:30.091769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f331fe-7c1c-4948-8267-0bc2f1938529 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:30.092218 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f331fe-7c1c-4948-8267-0bc2f1938529 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:30.098557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 679/1356] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:30.100103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb307f8-7dea-4315-a006-ac3bbb758bbd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:30.100585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb307f8-7dea-4315-a006-ac3bbb758bbd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:30.100867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb307f8-7dea-4315-a006-ac3bbb758bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:30.120608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-feb307f8-7dea-4315-a006-ac3bbb758bbd None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:30.121993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-feb307f8-7dea-4315-a006-ac3bbb758bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:30.133358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 678/1357] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:30.194151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f49133-b8d7-48aa-90c2-435e8ec7719c None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:30.194347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f49133-b8d7-48aa-90c2-435e8ec7719c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:30.194575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f49133-b8d7-48aa-90c2-435e8ec7719c None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:30.196203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 680/1358] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 4 09:07:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:30.217299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a705ba-95a0-4295-a04f-6bff0018544c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:30.217586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a705ba-95a0-4295-a04f-6bff0018544c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:30.218359 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a705ba-95a0-4295-a04f-6bff0018544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:30.277179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:30.286207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-64a705ba-95a0-4295-a04f-6bff0018544c None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:30.288030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-64a705ba-95a0-4295-a04f-6bff0018544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:30.375146 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 679/1359] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 4 09:07:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:07:30.383024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f49133-b8d7-48aa-90c2-435e8ec7719c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:30.397549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f49133-b8d7-48aa-90c2-435e8ec7719c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:30.403107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da129a-7bfc-4229-bbe7-b5e82a3c611d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:30.403883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da129a-7bfc-4229-bbe7-b5e82a3c611d service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:30.404260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da129a-7bfc-4229-bbe7-b5e82a3c611d service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:30.404633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da129a-7bfc-4229-bbe7-b5e82a3c611d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:30.413971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da129a-7bfc-4229-bbe7-b5e82a3c611d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:30.415691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da129a-7bfc-4229-bbe7-b5e82a3c611d service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:30.425758 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 681/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:32.645610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15058641-0a72-4637-adc1-5796361c6c64 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.646230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15058641-0a72-4637-adc1-5796361c6c64 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.646230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15058641-0a72-4637-adc1-5796361c6c64 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.648176 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 680/1361] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:32.652920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993a640-a100-4c7b-851e-8f2a3a47f053 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.653386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993a640-a100-4c7b-851e-8f2a3a47f053 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.654900 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993a640-a100-4c7b-851e-8f2a3a47f053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.673756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b993a640-a100-4c7b-851e-8f2a3a47f053 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:32.675189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b993a640-a100-4c7b-851e-8f2a3a47f053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:32.684802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 682/1362] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:32.730301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15058641-0a72-4637-adc1-5796361c6c64 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:32.745350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15058641-0a72-4637-adc1-5796361c6c64 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:32.751353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cc1b2-44e6-4a8c-b20a-c51772c8c7d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:32.752014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cc1b2-44e6-4a8c-b20a-c51772c8c7d1 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:32.752280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cc1b2-44e6-4a8c-b20a-c51772c8c7d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:32.752568 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cc1b2-44e6-4a8c-b20a-c51772c8c7d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:32.762742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cc1b2-44e6-4a8c-b20a-c51772c8c7d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:32.764354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cc1b2-44e6-4a8c-b20a-c51772c8c7d1 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:32.775338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 681/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:35.152770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9bd7f-5b40-4b9e-81ba-0c3db079fe78 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.153295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9bd7f-5b40-4b9e-81ba-0c3db079fe78 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.153295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9bd7f-5b40-4b9e-81ba-0c3db079fe78 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.155135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 683/1364] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:35.160561 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a7f65-c56f-49d7-a58b-c9c71ebb3a46 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.160979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a7f65-c56f-49d7-a58b-c9c71ebb3a46 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.161108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a7f65-c56f-49d7-a58b-c9c71ebb3a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.179129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-920a7f65-c56f-49d7-a58b-c9c71ebb3a46 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:35.180306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-920a7f65-c56f-49d7-a58b-c9c71ebb3a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:35.188368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 682/1365] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:35.233135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26ba8-4f9c-4296-abad-905c42ab13a6 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.233505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26ba8-4f9c-4296-abad-905c42ab13a6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.233854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26ba8-4f9c-4296-abad-905c42ab13a6 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.235322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 684/1366] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 4 09:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:07:35.240670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da4277-74f4-46e7-b2e3-7c875eb4ba6d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.240994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da4277-74f4-46e7-b2e3-7c875eb4ba6d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.241330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da4277-74f4-46e7-b2e3-7c875eb4ba6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.260069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-74da4277-74f4-46e7-b2e3-7c875eb4ba6d None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:35.261174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-74da4277-74f4-46e7-b2e3-7c875eb4ba6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:35.271961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 683/1367] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 4 09:07:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:35.279783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e26ba8-4f9c-4296-abad-905c42ab13a6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:35.295462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e26ba8-4f9c-4296-abad-905c42ab13a6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:35.301387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6d6fc-b34a-4812-92a4-abe28b94d7b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:35.302071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6d6fc-b34a-4812-92a4-abe28b94d7b9 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:35.302359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6d6fc-b34a-4812-92a4-abe28b94d7b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:35.302642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6d6fc-b34a-4812-92a4-abe28b94d7b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:35.310914 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6d6fc-b34a-4812-92a4-abe28b94d7b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:35.312539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6d6fc-b34a-4812-92a4-abe28b94d7b9 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:35.322387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 685/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:07:37.402353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055b5a6-716e-45af-aa53-ecf58347ad96 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:37.402952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055b5a6-716e-45af-aa53-ecf58347ad96 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:37.402952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055b5a6-716e-45af-aa53-ecf58347ad96 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:37.404243 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 684/1369] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:37.409848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4050f8-5a43-4ce3-9fad-4b890b3514d6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:37.410302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4050f8-5a43-4ce3-9fad-4b890b3514d6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:37.410389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4050f8-5a43-4ce3-9fad-4b890b3514d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:37.429996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-cd4050f8-5a43-4ce3-9fad-4b890b3514d6 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:37.431316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cd4050f8-5a43-4ce3-9fad-4b890b3514d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:37.440501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 686/1370] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:37.485405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055b5a6-716e-45af-aa53-ecf58347ad96 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:37.501492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055b5a6-716e-45af-aa53-ecf58347ad96 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:37.507731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d41245-7e1d-4341-97e1-4f23fcd2ebd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:37.508391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41245-7e1d-4341-97e1-4f23fcd2ebd9 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:37.508678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41245-7e1d-4341-97e1-4f23fcd2ebd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:37.508951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41245-7e1d-4341-97e1-4f23fcd2ebd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:37.518240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d41245-7e1d-4341-97e1-4f23fcd2ebd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:37.519829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d41245-7e1d-4341-97e1-4f23fcd2ebd9 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:37.529800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 685/1371] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:38.112636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763ff89-4a2d-4fcd-9e90-d29961c4df5f None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.112900 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763ff89-4a2d-4fcd-9e90-d29961c4df5f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.113449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763ff89-4a2d-4fcd-9e90-d29961c4df5f None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.115051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 687/1372] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:38.120657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ddd1-ddc4-4935-81a4-745393e281c6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.121164 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ddd1-ddc4-4935-81a4-745393e281c6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.121651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ddd1-ddc4-4935-81a4-745393e281c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.143628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d760ddd1-ddc4-4935-81a4-745393e281c6 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:38.145219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d760ddd1-ddc4-4935-81a4-745393e281c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:38.158913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 686/1373] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:38.517235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5089a1-25bd-449c-afd9-8986f67d44b3 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.517235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5089a1-25bd-449c-afd9-8986f67d44b3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.517235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5089a1-25bd-449c-afd9-8986f67d44b3 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.521101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 688/1374] 10.222.0.26 () {54 vars in 963 bytes} [Mon Jul 4 09:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:38.524138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efeb46-3c6d-4775-b955-6b5c7ead8b40 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.524138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efeb46-3c6d-4775-b955-6b5c7ead8b40 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.524138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efeb46-3c6d-4775-b955-6b5c7ead8b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.576003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d3efeb46-3c6d-4775-b955-6b5c7ead8b40 None None] MFA Rules not processed for user `e9f34c35a9ba4376a9b55ec5e23cf35f`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:38.577344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d3efeb46-3c6d-4775-b955-6b5c7ead8b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:38.641953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 687/1375] 10.222.0.26 () {58 vars in 1060 bytes} [Mon Jul 4 09:07:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:38.649922 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5089a1-25bd-449c-afd9-8986f67d44b3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:38.661927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5089a1-25bd-449c-afd9-8986f67d44b3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:38.666391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e59bb0-6580-45b7-b1ab-4c190b8b7d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:38.667176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59bb0-6580-45b7-b1ab-4c190b8b7d50 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:38.667927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59bb0-6580-45b7-b1ab-4c190b8b7d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:38.668307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59bb0-6580-45b7-b1ab-4c190b8b7d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:38.680029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e59bb0-6580-45b7-b1ab-4c190b8b7d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:38.681650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e59bb0-6580-45b7-b1ab-4c190b8b7d50 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:38.692521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 689/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:39.593043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21535b9d-a5e8-401f-8feb-3cfd0175cc85 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:39.593627 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21535b9d-a5e8-401f-8feb-3cfd0175cc85 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:39.593627 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21535b9d-a5e8-401f-8feb-3cfd0175cc85 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:39.595040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 688/1377] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:39.600452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d78295-33f9-40a1-888d-f52f060f08f4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:39.601427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d78295-33f9-40a1-888d-f52f060f08f4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:39.601427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d78295-33f9-40a1-888d-f52f060f08f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:39.620274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b4d78295-33f9-40a1-888d-f52f060f08f4 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:39.621468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b4d78295-33f9-40a1-888d-f52f060f08f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:39.630106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 690/1378] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:39.674346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21535b9d-a5e8-401f-8feb-3cfd0175cc85 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:39.690894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21535b9d-a5e8-401f-8feb-3cfd0175cc85 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:39.697319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e81783-a08a-406a-9118-a2084bdd1a9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:39.697990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e81783-a08a-406a-9118-a2084bdd1a9e service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:39.698277 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e81783-a08a-406a-9118-a2084bdd1a9e service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:39.698571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e81783-a08a-406a-9118-a2084bdd1a9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:39.707447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e81783-a08a-406a-9118-a2084bdd1a9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:39.709057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e81783-a08a-406a-9118-a2084bdd1a9e service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:39.713322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:39.721938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 689/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:41.784208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a20a6e-09aa-4ad9-8fcc-df16a4be34e5 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.785132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a20a6e-09aa-4ad9-8fcc-df16a4be34e5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.785520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a20a6e-09aa-4ad9-8fcc-df16a4be34e5 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.787101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 691/1380] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:41.792655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e7733-0e3b-4485-bc8b-f02d406acb96 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.793059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e7733-0e3b-4485-bc8b-f02d406acb96 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.793219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e7733-0e3b-4485-bc8b-f02d406acb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.812668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ad1e7733-0e3b-4485-bc8b-f02d406acb96 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:41.814171 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ad1e7733-0e3b-4485-bc8b-f02d406acb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:41.823473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 690/1381] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:41.868125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a20a6e-09aa-4ad9-8fcc-df16a4be34e5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:41.882366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a20a6e-09aa-4ad9-8fcc-df16a4be34e5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:41.888385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3541f6e-6174-48d6-afca-08e6d8609bec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:41.889032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3541f6e-6174-48d6-afca-08e6d8609bec service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:41.889342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3541f6e-6174-48d6-afca-08e6d8609bec service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:41.889617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3541f6e-6174-48d6-afca-08e6d8609bec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:41.898141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3541f6e-6174-48d6-afca-08e6d8609bec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:41.899736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3541f6e-6174-48d6-afca-08e6d8609bec service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:41.909135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 692/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:43.962241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403a69e-c401-478e-913a-f193b6419bd3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:43.962759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403a69e-c401-478e-913a-f193b6419bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:43.962759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403a69e-c401-478e-913a-f193b6419bd3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:43.964143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 691/1383] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:43.969575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f70ec1-c269-4db5-b4e0-d9a180c6441c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:43.970060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f70ec1-c269-4db5-b4e0-d9a180c6441c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:43.970163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f70ec1-c269-4db5-b4e0-d9a180c6441c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:43.988571 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-35f70ec1-c269-4db5-b4e0-d9a180c6441c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:43.989739 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-35f70ec1-c269-4db5-b4e0-d9a180c6441c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:43.998927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 693/1384] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:07:44.041119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403a69e-c401-478e-913a-f193b6419bd3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:44.056800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403a69e-c401-478e-913a-f193b6419bd3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:44.063540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c157a-cf78-48d1-b174-46ffe61c9c3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:44.064456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c157a-cf78-48d1-b174-46ffe61c9c3f service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:44.064968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c157a-cf78-48d1-b174-46ffe61c9c3f service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:44.065465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c157a-cf78-48d1-b174-46ffe61c9c3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:44.075192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c157a-cf78-48d1-b174-46ffe61c9c3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:44.077116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c157a-cf78-48d1-b174-46ffe61c9c3f service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:44.089536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 692/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:46.133242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61268c8b-aa4d-4451-ab4a-6f12a9a24d5c None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:46.133242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61268c8b-aa4d-4451-ab4a-6f12a9a24d5c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:46.136303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61268c8b-aa4d-4451-ab4a-6f12a9a24d5c None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:46.136303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 694/1386] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:46.140623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04411d5b-fb0d-4d15-9c9a-d423e5c0fe0f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:46.141422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04411d5b-fb0d-4d15-9c9a-d423e5c0fe0f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:46.141422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04411d5b-fb0d-4d15-9c9a-d423e5c0fe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:46.166799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-04411d5b-fb0d-4d15-9c9a-d423e5c0fe0f None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:46.168799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-04411d5b-fb0d-4d15-9c9a-d423e5c0fe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:46.184647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 693/1387] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:46.227980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61268c8b-aa4d-4451-ab4a-6f12a9a24d5c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:46.243801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61268c8b-aa4d-4451-ab4a-6f12a9a24d5c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:46.249484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37610a7-646a-4d81-94ed-9467e1076d2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:46.250188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37610a7-646a-4d81-94ed-9467e1076d2e service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:46.250469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37610a7-646a-4d81-94ed-9467e1076d2e service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:46.250743 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37610a7-646a-4d81-94ed-9467e1076d2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:46.259923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37610a7-646a-4d81-94ed-9467e1076d2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:46.261555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37610a7-646a-4d81-94ed-9467e1076d2e service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:46.284391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 695/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:48.412819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e09c0c-1a7a-45bd-b2f2-279cc790b6a8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:48.413489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e09c0c-1a7a-45bd-b2f2-279cc790b6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:48.413489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e09c0c-1a7a-45bd-b2f2-279cc790b6a8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:48.414875 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 694/1389] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:48.420630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c22c6d-b7cb-410d-a61a-e555b15c4b94 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:48.421110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c22c6d-b7cb-410d-a61a-e555b15c4b94 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:48.421225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c22c6d-b7cb-410d-a61a-e555b15c4b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:48.442210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e0c22c6d-b7cb-410d-a61a-e555b15c4b94 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:48.443775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e0c22c6d-b7cb-410d-a61a-e555b15c4b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:48.454681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 696/1390] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:48.502499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e09c0c-1a7a-45bd-b2f2-279cc790b6a8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:48.518055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e09c0c-1a7a-45bd-b2f2-279cc790b6a8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:48.524078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31a487-7fad-4d9b-8e77-5c2a53ceb4b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:48.524732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31a487-7fad-4d9b-8e77-5c2a53ceb4b4 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:48.525016 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31a487-7fad-4d9b-8e77-5c2a53ceb4b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:48.525294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31a487-7fad-4d9b-8e77-5c2a53ceb4b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:48.534519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31a487-7fad-4d9b-8e77-5c2a53ceb4b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:48.536133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31a487-7fad-4d9b-8e77-5c2a53ceb4b4 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:48.547150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 695/1391] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:50.646050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a6825-fd8c-4871-aa34-2a59a69839ea None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:50.646650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a6825-fd8c-4871-aa34-2a59a69839ea None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:50.646650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a6825-fd8c-4871-aa34-2a59a69839ea None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:50.648325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 697/1392] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:50.654458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd55c5-fa38-4ed8-9346-450e868d5705 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:50.654951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd55c5-fa38-4ed8-9346-450e868d5705 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:50.655048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd55c5-fa38-4ed8-9346-450e868d5705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:50.675305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f7cd55c5-fa38-4ed8-9346-450e868d5705 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:50.676593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f7cd55c5-fa38-4ed8-9346-450e868d5705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:50.685772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 696/1393] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:50.732616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a6825-fd8c-4871-aa34-2a59a69839ea None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:50.747682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a6825-fd8c-4871-aa34-2a59a69839ea None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:50.753511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d569fcc-79df-4c07-869d-37ebddc4f999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:50.754201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d569fcc-79df-4c07-869d-37ebddc4f999 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:50.754491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d569fcc-79df-4c07-869d-37ebddc4f999 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:50.754772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d569fcc-79df-4c07-869d-37ebddc4f999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:50.764009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d569fcc-79df-4c07-869d-37ebddc4f999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:50.765634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d569fcc-79df-4c07-869d-37ebddc4f999 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:50.775804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 698/1394] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:53.068383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b2140-c801-47aa-bdc0-f7010ebcdf47 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:53.068938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b2140-c801-47aa-bdc0-f7010ebcdf47 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:53.069033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b2140-c801-47aa-bdc0-f7010ebcdf47 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:53.070698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 697/1395] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:53.076304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce2dfa-3717-49bd-8cd8-0611b60dc097 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:53.076889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce2dfa-3717-49bd-8cd8-0611b60dc097 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:53.076889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce2dfa-3717-49bd-8cd8-0611b60dc097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:53.097132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ddce2dfa-3717-49bd-8cd8-0611b60dc097 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:53.098371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ddce2dfa-3717-49bd-8cd8-0611b60dc097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:53.107265 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 699/1396] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:53.152042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b2140-c801-47aa-bdc0-f7010ebcdf47 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:53.167458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b2140-c801-47aa-bdc0-f7010ebcdf47 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:53.173476 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e4978-82a4-42ff-95f3-bd8835a421b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:53.174181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4978-82a4-42ff-95f3-bd8835a421b4 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:53.174469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4978-82a4-42ff-95f3-bd8835a421b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:53.174750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4978-82a4-42ff-95f3-bd8835a421b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:53.183946 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e4978-82a4-42ff-95f3-bd8835a421b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:53.185557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e4978-82a4-42ff-95f3-bd8835a421b4 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:53.195622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 698/1397] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:54.022953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97610758-425a-4e70-ab02-607d76eda5f3 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:54.023357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97610758-425a-4e70-ab02-607d76eda5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:54.023469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97610758-425a-4e70-ab02-607d76eda5f3 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:54.024876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 700/1398] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:54.030177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13665a89-849d-423e-8b7f-9a76e55a0dc7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:54.030462 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13665a89-849d-423e-8b7f-9a76e55a0dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:54.030740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13665a89-849d-423e-8b7f-9a76e55a0dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:54.049505 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-13665a89-849d-423e-8b7f-9a76e55a0dc7 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:54.050639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-13665a89-849d-423e-8b7f-9a76e55a0dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:54.059035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 699/1399] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:54.112699 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981b770-e0d2-4c95-9e5f-6005dce76ebb None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:54.113177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981b770-e0d2-4c95-9e5f-6005dce76ebb None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:54.113409 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981b770-e0d2-4c95-9e5f-6005dce76ebb None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:54.114793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 701/1400] 10.222.0.26 () {54 vars in 962 bytes} [Mon Jul 4 09:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:54.120075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911a754-5d26-4a30-8cde-f814958c5e3d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:54.120882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911a754-5d26-4a30-8cde-f814958c5e3d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:54.121163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911a754-5d26-4a30-8cde-f814958c5e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:54.170274 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2911a754-5d26-4a30-8cde-f814958c5e3d None None] MFA Rules not processed for user `107a90e384504e8d80264d6bf20f1d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:54.171464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2911a754-5d26-4a30-8cde-f814958c5e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:54.233482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 700/1401] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Jul 4 09:07:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:07:54.241338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981b770-e0d2-4c95-9e5f-6005dce76ebb None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:54.253586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981b770-e0d2-4c95-9e5f-6005dce76ebb None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:54.258391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575975a1-6fa2-4597-a526-493b4005a0b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a90e384504e8d80264d6bf20f1d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:54.259151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575975a1-6fa2-4597-a526-493b4005a0b4 service glance] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:54.259505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575975a1-6fa2-4597-a526-493b4005a0b4 service glance] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:54.259859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575975a1-6fa2-4597-a526-493b4005a0b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:54.270547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575975a1-6fa2-4597-a526-493b4005a0b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:54.272035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575975a1-6fa2-4597-a526-493b4005a0b4 service glance] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:54.282724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 702/1402] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:07:55.001968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d1182-6c74-498b-8581-aa68107fe12d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:55.002259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d1182-6c74-498b-8581-aa68107fe12d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:55.002544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d1182-6c74-498b-8581-aa68107fe12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:55.053492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-383d1182-6c74-498b-8581-aa68107fe12d None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:55.054731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-383d1182-6c74-498b-8581-aa68107fe12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:55.100673 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:55.125989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 701/1403] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:07:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:55.170185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff15d12-0352-4ede-b6fd-d2fedf4886b5 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:55.170686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff15d12-0352-4ede-b6fd-d2fedf4886b5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:55.170867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff15d12-0352-4ede-b6fd-d2fedf4886b5 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:55.172361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 703/1404] 10.222.0.26 () {54 vars in 942 bytes} [Mon Jul 4 09:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:55.178671 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed5144-831a-4785-b1e7-8a007e35f122 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:55.179035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed5144-831a-4785-b1e7-8a007e35f122 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:55.179401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed5144-831a-4785-b1e7-8a007e35f122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:55.229075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f0ed5144-831a-4785-b1e7-8a007e35f122 None None] MFA Rules not processed for user `8cbb65dbfadf4c63a272f5318efa875a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:55.230330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f0ed5144-831a-4785-b1e7-8a007e35f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:55.282259 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6af087-b167-4495-9211-26dafab8b60f None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:55.282447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6af087-b167-4495-9211-26dafab8b60f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:55.282733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6af087-b167-4495-9211-26dafab8b60f None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:55.284115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 704/1405] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:07:55.289369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e8a08-6e65-41d4-8edb-3ccc49ed8375 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:55.289691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e8a08-6e65-41d4-8edb-3ccc49ed8375 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:55.290115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e8a08-6e65-41d4-8edb-3ccc49ed8375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:55.303764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 702/1406] 10.222.0.26 () {58 vars in 1039 bytes} [Mon Jul 4 09:07:55 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:55.308595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-db9e8a08-6e65-41d4-8edb-3ccc49ed8375 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:07:55.309811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-db9e8a08-6e65-41d4-8edb-3ccc49ed8375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:07:55.313511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed5144-831a-4785-b1e7-8a007e35f122 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:55.318502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 705/1407] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:07:55.325338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed5144-831a-4785-b1e7-8a007e35f122 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:55.330604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2616658-a4bc-40cf-9330-7f72b2f0b622 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:55.331244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2616658-a4bc-40cf-9330-7f72b2f0b622 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:55.331523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2616658-a4bc-40cf-9330-7f72b2f0b622 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:55.331791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2616658-a4bc-40cf-9330-7f72b2f0b622 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:55.340046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2616658-a4bc-40cf-9330-7f72b2f0b622 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:55.341631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2616658-a4bc-40cf-9330-7f72b2f0b622 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:55.349538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 703/1408] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:07:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:07:55.362844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e8a08-6e65-41d4-8edb-3ccc49ed8375 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:07:55.370692 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=64 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:07:55.378248 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e8a08-6e65-41d4-8edb-3ccc49ed8375 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:07:55.385063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42bebf-2ab3-462f-8a06-bd924574447e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:07:55.385722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42bebf-2ab3-462f-8a06-bd924574447e service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:07:55.386045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42bebf-2ab3-462f-8a06-bd924574447e service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:07:55.386328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42bebf-2ab3-462f-8a06-bd924574447e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:07:55.395707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42bebf-2ab3-462f-8a06-bd924574447e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:07:55.397304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42bebf-2ab3-462f-8a06-bd924574447e service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:07:55.408577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 706/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:07:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:05.349237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=59 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:05.403744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:17.353757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18def272-f126-4500-bc7e-17469967aafc None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:17.354418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18def272-f126-4500-bc7e-17469967aafc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:17.354418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18def272-f126-4500-bc7e-17469967aafc None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:17.355758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 704/1410] 10.222.0.26 () {54 vars in 918 bytes} [Mon Jul 4 09:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:17.365820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa49e6-e526-423b-a32a-09d34d2788b9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:17.366555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa49e6-e526-423b-a32a-09d34d2788b9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:17.366555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa49e6-e526-423b-a32a-09d34d2788b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:17.395447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e9aa49e6-e526-423b-a32a-09d34d2788b9 None None] MFA Rules not processed for user `aff2305249a242639e1415094bf9df33`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:17.397081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e9aa49e6-e526-423b-a32a-09d34d2788b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:17.406284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 707/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Jul 4 09:08:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:17.422103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3a3c7-c08a-4839-9f25-63bbea02a0bb None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:17.422397 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3a3c7-c08a-4839-9f25-63bbea02a0bb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:17.422672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3a3c7-c08a-4839-9f25-63bbea02a0bb None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:17.425235 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 705/1412] 10.222.0.26 () {54 vars in 918 bytes} [Mon Jul 4 09:08:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:17.431065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c8f74-1d67-4766-bd0d-547d64041612 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:17.431351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c8f74-1d67-4766-bd0d-547d64041612 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:17.431629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c8f74-1d67-4766-bd0d-547d64041612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:17.450748 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-124c8f74-1d67-4766-bd0d-547d64041612 None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:17.451933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-124c8f74-1d67-4766-bd0d-547d64041612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:17.462938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 708/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Jul 4 09:08:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:08:19.961259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7993c78-5b4d-454d-94c6-11033c465e9b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:19.961829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7993c78-5b4d-454d-94c6-11033c465e9b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:19.961829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7993c78-5b4d-454d-94c6-11033c465e9b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:19.963117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:19.969355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e9e8e-7aed-4b84-978b-50a8df1349f8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:19.970098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e9e8e-7aed-4b84-978b-50a8df1349f8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:19.970278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e9e8e-7aed-4b84-978b-50a8df1349f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:19.991389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b32e9e8e-7aed-4b84-978b-50a8df1349f8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:19.992589 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b32e9e8e-7aed-4b84-978b-50a8df1349f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:20.001068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:20.047800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7993c78-5b4d-454d-94c6-11033c465e9b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:20.063436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7993c78-5b4d-454d-94c6-11033c465e9b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:20.069160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fcad5-dc30-4ef4-bca7-d69ed142b8f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:20.069802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fcad5-dc30-4ef4-bca7-d69ed142b8f8 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:20.070110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fcad5-dc30-4ef4-bca7-d69ed142b8f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:20.070392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fcad5-dc30-4ef4-bca7-d69ed142b8f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:20.078477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fcad5-dc30-4ef4-bca7-d69ed142b8f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:20.080057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fcad5-dc30-4ef4-bca7-d69ed142b8f8 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:20.089991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 707/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:20.980714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6fbc-b550-4104-93cb-9816e7615eaf None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:20.981335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6fbc-b550-4104-93cb-9816e7615eaf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:20.981335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6fbc-b550-4104-93cb-9816e7615eaf None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:20.982691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:20.988839 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39313809-2b6e-4135-992a-da4cfb54779e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:20.989571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39313809-2b6e-4135-992a-da4cfb54779e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:20.990038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39313809-2b6e-4135-992a-da4cfb54779e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:21.010854 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-39313809-2b6e-4135-992a-da4cfb54779e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:21.012394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-39313809-2b6e-4135-992a-da4cfb54779e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:21.022229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:21.180562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e6fbc-b550-4104-93cb-9816e7615eaf None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:21.204495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e6fbc-b550-4104-93cb-9816e7615eaf None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:21.212115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfe936-ed79-4277-a3f4-0f4c1cc4b53a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:21.212765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfe936-ed79-4277-a3f4-0f4c1cc4b53a service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:21.213050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfe936-ed79-4277-a3f4-0f4c1cc4b53a service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:21.213326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfe936-ed79-4277-a3f4-0f4c1cc4b53a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:21.222604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfe936-ed79-4277-a3f4-0f4c1cc4b53a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:21.224366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfe936-ed79-4277-a3f4-0f4c1cc4b53a service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:21.234838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 711/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:22.120036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9ccb1-25e0-4f6f-8e25-107003c9f599 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:22.120629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9ccb1-25e0-4f6f-8e25-107003c9f599 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:22.120629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9ccb1-25e0-4f6f-8e25-107003c9f599 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:22.122051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:22.127168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22237d-9afe-4ce6-a1be-73747aca8de4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:22.127634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22237d-9afe-4ce6-a1be-73747aca8de4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:22.127725 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22237d-9afe-4ce6-a1be-73747aca8de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:22.147044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3a22237d-9afe-4ce6-a1be-73747aca8de4 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:22.148224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3a22237d-9afe-4ce6-a1be-73747aca8de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:22.156781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:22.317567 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9ccb1-25e0-4f6f-8e25-107003c9f599 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:22.334443 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9ccb1-25e0-4f6f-8e25-107003c9f599 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:22.340782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337d219-d306-4c99-b5d9-3a2e282d2611 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:22.341436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337d219-d306-4c99-b5d9-3a2e282d2611 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:22.341717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337d219-d306-4c99-b5d9-3a2e282d2611 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:22.342027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337d219-d306-4c99-b5d9-3a2e282d2611 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:22.351097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337d219-d306-4c99-b5d9-3a2e282d2611 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:22.352738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337d219-d306-4c99-b5d9-3a2e282d2611 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:22.362536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 710/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:23.211772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998823d-6667-40b3-8f30-593f69f9d984 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:23.212336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998823d-6667-40b3-8f30-593f69f9d984 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:23.212336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998823d-6667-40b3-8f30-593f69f9d984 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:23.213693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:23.219788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844dc0f-8531-4e3d-893d-65b268e671b5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:23.220266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844dc0f-8531-4e3d-893d-65b268e671b5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:23.220374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844dc0f-8531-4e3d-893d-65b268e671b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:23.244893 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0844dc0f-8531-4e3d-893d-65b268e671b5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:23.246088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0844dc0f-8531-4e3d-893d-65b268e671b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:23.254459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:23.406729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998823d-6667-40b3-8f30-593f69f9d984 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:23.424007 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998823d-6667-40b3-8f30-593f69f9d984 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:23.431341 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12501b-2fdd-484a-9207-e98ad308167b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:23.431979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12501b-2fdd-484a-9207-e98ad308167b service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:23.432260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12501b-2fdd-484a-9207-e98ad308167b service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:23.432532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12501b-2fdd-484a-9207-e98ad308167b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:23.441157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12501b-2fdd-484a-9207-e98ad308167b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:23.443144 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12501b-2fdd-484a-9207-e98ad308167b service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:23.452678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 714/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:24.328280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceffef-cf92-464b-8317-892b73954e41 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:24.328874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceffef-cf92-464b-8317-892b73954e41 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:24.328874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceffef-cf92-464b-8317-892b73954e41 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:24.330259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:24.336530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3c175-6876-4989-9de2-ae515a6a9f59 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:24.337317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3c175-6876-4989-9de2-ae515a6a9f59 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:24.337487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3c175-6876-4989-9de2-ae515a6a9f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:24.361444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f9f3c175-6876-4989-9de2-ae515a6a9f59 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:24.362706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f9f3c175-6876-4989-9de2-ae515a6a9f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:24.372016 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:24.524648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceffef-cf92-464b-8317-892b73954e41 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:24.546194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceffef-cf92-464b-8317-892b73954e41 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:24.553271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce18a6a-4ef6-4ea3-9687-9616d76b51a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:24.553908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce18a6a-4ef6-4ea3-9687-9616d76b51a1 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:24.554228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce18a6a-4ef6-4ea3-9687-9616d76b51a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:24.554501 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce18a6a-4ef6-4ea3-9687-9616d76b51a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:24.562801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce18a6a-4ef6-4ea3-9687-9616d76b51a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:24.564386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce18a6a-4ef6-4ea3-9687-9616d76b51a1 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:24.574746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 713/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:25.383129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05392d95-db5b-4d8b-9cb9-890db3d3fd33 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:25.384153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05392d95-db5b-4d8b-9cb9-890db3d3fd33 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:25.384153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05392d95-db5b-4d8b-9cb9-890db3d3fd33 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:25.386324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:08:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:25.392684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3be8-96cd-4bec-b38a-813930d093dd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:25.393145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3be8-96cd-4bec-b38a-813930d093dd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:25.393404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3be8-96cd-4bec-b38a-813930d093dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:25.414799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3f9a3be8-96cd-4bec-b38a-813930d093dd None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:25.415896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3f9a3be8-96cd-4bec-b38a-813930d093dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:25.424210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:25.575992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05392d95-db5b-4d8b-9cb9-890db3d3fd33 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:25.595625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05392d95-db5b-4d8b-9cb9-890db3d3fd33 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:25.605846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b78e0-a534-4349-8a02-3b1e2b14c100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:25.606825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b78e0-a534-4349-8a02-3b1e2b14c100 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:25.607114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b78e0-a534-4349-8a02-3b1e2b14c100 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:25.607446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b78e0-a534-4349-8a02-3b1e2b14c100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:25.619187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b78e0-a534-4349-8a02-3b1e2b14c100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:25.620795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b78e0-a534-4349-8a02-3b1e2b14c100 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:25.631213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 717/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:26.438152 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbc55e-6552-4299-ba30-a671bfb9885d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:26.438727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbc55e-6552-4299-ba30-a671bfb9885d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:26.438727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbc55e-6552-4299-ba30-a671bfb9885d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:26.440018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:26.446411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f377-ba84-4ced-ba33-60b889d0d6d0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:26.447178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f377-ba84-4ced-ba33-60b889d0d6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:26.447373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f377-ba84-4ced-ba33-60b889d0d6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:26.470385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1563f377-ba84-4ced-ba33-60b889d0d6d0 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:26.471610 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1563f377-ba84-4ced-ba33-60b889d0d6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:26.482281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:26.634258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbc55e-6552-4299-ba30-a671bfb9885d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:26.650967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbc55e-6552-4299-ba30-a671bfb9885d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:26.657153 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ab4d9-6f14-4795-87fa-bc8ff8bbafa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:26.657846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ab4d9-6f14-4795-87fa-bc8ff8bbafa3 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:26.658166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ab4d9-6f14-4795-87fa-bc8ff8bbafa3 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:26.658440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ab4d9-6f14-4795-87fa-bc8ff8bbafa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:26.667018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ab4d9-6f14-4795-87fa-bc8ff8bbafa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:26.668604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ab4d9-6f14-4795-87fa-bc8ff8bbafa3 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:26.678243 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 716/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:27.468694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f9e70-d29e-460c-82a1-06a40fffb2f5 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:27.469559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f9e70-d29e-460c-82a1-06a40fffb2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:27.469559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f9e70-d29e-460c-82a1-06a40fffb2f5 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:27.471734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:08:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:27.478659 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a31e9-d05c-454c-a055-1dca0b49d62a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:27.479133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a31e9-d05c-454c-a055-1dca0b49d62a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:27.479445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a31e9-d05c-454c-a055-1dca0b49d62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:27.507510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-116a31e9-d05c-454c-a055-1dca0b49d62a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:27.508692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-116a31e9-d05c-454c-a055-1dca0b49d62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:27.521366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:27.673785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f9e70-d29e-460c-82a1-06a40fffb2f5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:27.693945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f9e70-d29e-460c-82a1-06a40fffb2f5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:27.704795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b72c5a-0c39-48ac-8527-cda123150104 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:27.705735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b72c5a-0c39-48ac-8527-cda123150104 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:27.706183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b72c5a-0c39-48ac-8527-cda123150104 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:27.706587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b72c5a-0c39-48ac-8527-cda123150104 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:27.716194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b72c5a-0c39-48ac-8527-cda123150104 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:27.718375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b72c5a-0c39-48ac-8527-cda123150104 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:27.728710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 720/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:28.531297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4816c0d-55d9-452e-8afb-d213621a6136 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:28.531859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4816c0d-55d9-452e-8afb-d213621a6136 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:28.531859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4816c0d-55d9-452e-8afb-d213621a6136 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:28.533144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:28.539282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54989-e28a-44b0-976d-5b3198c6b425 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:28.540086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54989-e28a-44b0-976d-5b3198c6b425 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:28.540259 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54989-e28a-44b0-976d-5b3198c6b425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:28.561830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e2d54989-e28a-44b0-976d-5b3198c6b425 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:28.563026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e2d54989-e28a-44b0-976d-5b3198c6b425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:28.571406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:28.724549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4816c0d-55d9-452e-8afb-d213621a6136 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:28.741322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4816c0d-55d9-452e-8afb-d213621a6136 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:28.747478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed92b7d-1746-4513-b608-7574c4e33e55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:28.748098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed92b7d-1746-4513-b608-7574c4e33e55 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:28.748384 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed92b7d-1746-4513-b608-7574c4e33e55 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:28.748666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed92b7d-1746-4513-b608-7574c4e33e55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:28.757301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed92b7d-1746-4513-b608-7574c4e33e55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:28.758931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed92b7d-1746-4513-b608-7574c4e33e55 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:28.768999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 719/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:30.705785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41be9e-af29-4a5b-81e3-1eb5feb976e1 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:30.706772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41be9e-af29-4a5b-81e3-1eb5feb976e1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:30.706935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41be9e-af29-4a5b-81e3-1eb5feb976e1 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:30.708791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:08:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:30.714210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f72136-9d14-4e6e-8c0e-c857ba856432 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:30.714662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f72136-9d14-4e6e-8c0e-c857ba856432 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:30.714757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f72136-9d14-4e6e-8c0e-c857ba856432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:30.733747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-75f72136-9d14-4e6e-8c0e-c857ba856432 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:30.734951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-75f72136-9d14-4e6e-8c0e-c857ba856432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:30.744608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:30.782864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41be9e-af29-4a5b-81e3-1eb5feb976e1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:30.794951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41be9e-af29-4a5b-81e3-1eb5feb976e1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:30.798995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbe9fe-4455-466c-8e4c-ba7f9e197073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:30.799633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbe9fe-4455-466c-8e4c-ba7f9e197073 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:30.799912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbe9fe-4455-466c-8e4c-ba7f9e197073 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:30.800186 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbe9fe-4455-466c-8e4c-ba7f9e197073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:30.810635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbe9fe-4455-466c-8e4c-ba7f9e197073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:30.812090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbe9fe-4455-466c-8e4c-ba7f9e197073 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:30.821672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 723/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:32.987098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04265ae-7230-44f6-8ae4-66d762e6c3b7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:32.987676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04265ae-7230-44f6-8ae4-66d762e6c3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:32.987676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04265ae-7230-44f6-8ae4-66d762e6c3b7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:32.989120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:32.995787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bfb68-675d-48c5-bbd5-5096e73f1441 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:32.996241 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bfb68-675d-48c5-bbd5-5096e73f1441 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:32.996343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bfb68-675d-48c5-bbd5-5096e73f1441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:33.027389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ce8bfb68-675d-48c5-bbd5-5096e73f1441 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:33.028573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ce8bfb68-675d-48c5-bbd5-5096e73f1441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:33.037656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:33.076163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04265ae-7230-44f6-8ae4-66d762e6c3b7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:33.088151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04265ae-7230-44f6-8ae4-66d762e6c3b7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:33.093177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86113fbf-7178-45a5-b57a-5205613b6cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:33.093843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86113fbf-7178-45a5-b57a-5205613b6cdb service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:33.094147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86113fbf-7178-45a5-b57a-5205613b6cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:33.094423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86113fbf-7178-45a5-b57a-5205613b6cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:33.105264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86113fbf-7178-45a5-b57a-5205613b6cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:33.106733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86113fbf-7178-45a5-b57a-5205613b6cdb service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:33.117313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 722/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:35.440184 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc030c73-4d47-43ba-9efd-94795555eada None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:35.441072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc030c73-4d47-43ba-9efd-94795555eada None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:35.441261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc030c73-4d47-43ba-9efd-94795555eada None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:35.443802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:08:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:35.452942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989d92e-be18-40be-8b1c-9924f5c4a7d8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:35.453692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989d92e-be18-40be-8b1c-9924f5c4a7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:35.453872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989d92e-be18-40be-8b1c-9924f5c4a7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:35.475295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c989d92e-be18-40be-8b1c-9924f5c4a7d8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:35.477229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c989d92e-be18-40be-8b1c-9924f5c4a7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:35.490754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:35.534456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc030c73-4d47-43ba-9efd-94795555eada None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:35.547365 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc030c73-4d47-43ba-9efd-94795555eada None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:35.551625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127d7aa-f00e-44a4-b76c-81b34ae60c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:35.552340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127d7aa-f00e-44a4-b76c-81b34ae60c2d service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:35.552620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127d7aa-f00e-44a4-b76c-81b34ae60c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:35.552897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127d7aa-f00e-44a4-b76c-81b34ae60c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:35.562616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127d7aa-f00e-44a4-b76c-81b34ae60c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:35.564052 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127d7aa-f00e-44a4-b76c-81b34ae60c2d service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:35.573595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 726/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:37.696813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9d28f-33a0-46b5-b451-3dedece2b6d3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:37.697715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9d28f-33a0-46b5-b451-3dedece2b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:37.697829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9d28f-33a0-46b5-b451-3dedece2b6d3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:37.700024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:08:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:37.708672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb6d1-22ac-4910-8a80-7df6f3e4a168 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:37.709111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb6d1-22ac-4910-8a80-7df6f3e4a168 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:37.709220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb6d1-22ac-4910-8a80-7df6f3e4a168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:37.728462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-caccb6d1-22ac-4910-8a80-7df6f3e4a168 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:37.729659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-caccb6d1-22ac-4910-8a80-7df6f3e4a168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:37.738630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:37.780498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9d28f-33a0-46b5-b451-3dedece2b6d3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:37.793850 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9d28f-33a0-46b5-b451-3dedece2b6d3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:37.798837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60562ba6-f77e-44c0-a130-1ec6fa353dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:37.799473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562ba6-f77e-44c0-a130-1ec6fa353dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:37.799759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562ba6-f77e-44c0-a130-1ec6fa353dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:37.800032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562ba6-f77e-44c0-a130-1ec6fa353dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:37.809699 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60562ba6-f77e-44c0-a130-1ec6fa353dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:37.811164 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60562ba6-f77e-44c0-a130-1ec6fa353dc4 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:37.821232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 725/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:39.931634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f01ac-eacc-487e-b2d7-7310bcd77503 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:39.932487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f01ac-eacc-487e-b2d7-7310bcd77503 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:39.932487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f01ac-eacc-487e-b2d7-7310bcd77503 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:39.934747 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:08:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:39.944252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082f7c1-0b5a-4a16-ba0e-c812776437a8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:39.944692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082f7c1-0b5a-4a16-ba0e-c812776437a8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:39.944820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082f7c1-0b5a-4a16-ba0e-c812776437a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:39.968068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f082f7c1-0b5a-4a16-ba0e-c812776437a8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:39.969256 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f082f7c1-0b5a-4a16-ba0e-c812776437a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:39.978186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:40.016932 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f01ac-eacc-487e-b2d7-7310bcd77503 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:40.036626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f01ac-eacc-487e-b2d7-7310bcd77503 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:40.041154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f5b3b-1bd9-4df8-8edf-856efacee680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:40.041832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f5b3b-1bd9-4df8-8edf-856efacee680 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:40.042150 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f5b3b-1bd9-4df8-8edf-856efacee680 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:40.042430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f5b3b-1bd9-4df8-8edf-856efacee680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:40.052338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f5b3b-1bd9-4df8-8edf-856efacee680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:40.053768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f5b3b-1bd9-4df8-8edf-856efacee680 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:40.063467 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 729/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:42.142417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c23ed7-9b7a-4fc6-ba94-56b42bc8177b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:42.143035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c23ed7-9b7a-4fc6-ba94-56b42bc8177b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:42.143035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c23ed7-9b7a-4fc6-ba94-56b42bc8177b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:42.144368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:42.150115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795f43e-9719-4e5e-a1f2-f701cafa46f1 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:42.150560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795f43e-9719-4e5e-a1f2-f701cafa46f1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:42.150687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795f43e-9719-4e5e-a1f2-f701cafa46f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:42.177353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d795f43e-9719-4e5e-a1f2-f701cafa46f1 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:42.178587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d795f43e-9719-4e5e-a1f2-f701cafa46f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:42.188420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:42.227240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c23ed7-9b7a-4fc6-ba94-56b42bc8177b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:42.245034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c23ed7-9b7a-4fc6-ba94-56b42bc8177b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:42.250761 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f625a-b65d-4a9d-9a38-85507f28c38a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:42.251760 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f625a-b65d-4a9d-9a38-85507f28c38a service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:42.252164 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f625a-b65d-4a9d-9a38-85507f28c38a service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:42.252506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f625a-b65d-4a9d-9a38-85507f28c38a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:42.265361 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f625a-b65d-4a9d-9a38-85507f28c38a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:42.266858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f625a-b65d-4a9d-9a38-85507f28c38a service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:42.278628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 728/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:44.355542 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfa137-6a93-43a4-98cc-6eb0b1fc2519 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:44.356104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfa137-6a93-43a4-98cc-6eb0b1fc2519 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:44.356104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfa137-6a93-43a4-98cc-6eb0b1fc2519 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:44.358391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:08:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:44.364244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20b029-6892-44e3-aff0-5c7733670799 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:44.364673 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20b029-6892-44e3-aff0-5c7733670799 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:44.364803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20b029-6892-44e3-aff0-5c7733670799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:44.382824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2b20b029-6892-44e3-aff0-5c7733670799 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:44.384028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2b20b029-6892-44e3-aff0-5c7733670799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:44.392614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:44.430965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfa137-6a93-43a4-98cc-6eb0b1fc2519 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:44.449243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfa137-6a93-43a4-98cc-6eb0b1fc2519 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:44.455529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af740cc-a050-40ae-8115-06db64fb2e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:44.456507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af740cc-a050-40ae-8115-06db64fb2e49 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:44.457050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af740cc-a050-40ae-8115-06db64fb2e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:44.457461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af740cc-a050-40ae-8115-06db64fb2e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:44.467380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af740cc-a050-40ae-8115-06db64fb2e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:44.468829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af740cc-a050-40ae-8115-06db64fb2e49 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:44.479452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 732/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:46.531825 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744e3f-e620-4642-8f59-78badd822d20 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:46.532390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744e3f-e620-4642-8f59-78badd822d20 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:46.532390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60744e3f-e620-4642-8f59-78badd822d20 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:46.533696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:46.540251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942797c-c66e-47de-875b-18e92bb5cf19 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:46.540991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942797c-c66e-47de-875b-18e92bb5cf19 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:46.541310 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942797c-c66e-47de-875b-18e92bb5cf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:46.551461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=79 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:46.565243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e942797c-c66e-47de-875b-18e92bb5cf19 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:46.566490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e942797c-c66e-47de-875b-18e92bb5cf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:46.575827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:46.612469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60744e3f-e620-4642-8f59-78badd822d20 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:46.618254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=73 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:08:46.625213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60744e3f-e620-4642-8f59-78badd822d20 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:46.628710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e815f61-4294-4c66-9e34-a793e1ac04aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:46.629349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815f61-4294-4c66-9e34-a793e1ac04aa service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:46.629630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815f61-4294-4c66-9e34-a793e1ac04aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:46.629936 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e815f61-4294-4c66-9e34-a793e1ac04aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:46.639917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e815f61-4294-4c66-9e34-a793e1ac04aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:46.641336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e815f61-4294-4c66-9e34-a793e1ac04aa service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:46.651374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 731/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:08:48.680819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b0ce-ab98-4c45-98b4-d43967ef76ac None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:48.681695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b0ce-ab98-4c45-98b4-d43967ef76ac None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:48.681695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b0ce-ab98-4c45-98b4-d43967ef76ac None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:48.684064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:08:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:48.689610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e57d2-0875-458b-9675-0e013dacc172 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:48.690076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e57d2-0875-458b-9675-0e013dacc172 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:48.690169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e57d2-0875-458b-9675-0e013dacc172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:48.710153 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a96e57d2-0875-458b-9675-0e013dacc172 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:48.711352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a96e57d2-0875-458b-9675-0e013dacc172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:48.719720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:08:48.756045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25b0ce-ab98-4c45-98b4-d43967ef76ac None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:48.772639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25b0ce-ab98-4c45-98b4-d43967ef76ac None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:48.776466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7364e09-30da-4fc6-a4f0-77306fe720f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:48.777102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7364e09-30da-4fc6-a4f0-77306fe720f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:48.777380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7364e09-30da-4fc6-a4f0-77306fe720f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:48.777648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7364e09-30da-4fc6-a4f0-77306fe720f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:48.787234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7364e09-30da-4fc6-a4f0-77306fe720f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:48.788697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7364e09-30da-4fc6-a4f0-77306fe720f3 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:48.798278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 735/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:08:50.904443 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f990-8ecc-4bdc-87d0-eb67c93cbca2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:50.905186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f990-8ecc-4bdc-87d0-eb67c93cbca2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:50.905186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f990-8ecc-4bdc-87d0-eb67c93cbca2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:50.906801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:08:50.913865 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55508a9f-6406-473f-bf94-069a977e3322 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:50.914620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55508a9f-6406-473f-bf94-069a977e3322 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:50.914818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55508a9f-6406-473f-bf94-069a977e3322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:50.937327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-55508a9f-6406-473f-bf94-069a977e3322 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:08:50.938558 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-55508a9f-6406-473f-bf94-069a977e3322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:08:50.947975 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:08:50.983968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f990-8ecc-4bdc-87d0-eb67c93cbca2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:08:50.995980 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f990-8ecc-4bdc-87d0-eb67c93cbca2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:08:50.999894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a3462-bc79-4e00-9700-7b9ae542ff71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:08:51.000561 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a3462-bc79-4e00-9700-7b9ae542ff71 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:08:51.000840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a3462-bc79-4e00-9700-7b9ae542ff71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:08:51.001109 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a3462-bc79-4e00-9700-7b9ae542ff71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:08:51.011100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a3462-bc79-4e00-9700-7b9ae542ff71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:08:51.012537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a3462-bc79-4e00-9700-7b9ae542ff71 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:08:51.022087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 734/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:09:00.932071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=11 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:01.017921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=14 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:10.786521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d7b26-7d01-42a2-8442-9d84f85b836d None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:10.787482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d7b26-7d01-42a2-8442-9d84f85b836d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:10.787482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d7b26-7d01-42a2-8442-9d84f85b836d None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:10.789788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 737/1471] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:09:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:10.798299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd96e07-4bb9-4de7-b34b-b6ca8a561783 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:10.799071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd96e07-4bb9-4de7-b34b-b6ca8a561783 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:10.799212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd96e07-4bb9-4de7-b34b-b6ca8a561783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:10.824867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0fd96e07-4bb9-4de7-b34b-b6ca8a561783 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:10.826091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0fd96e07-4bb9-4de7-b34b-b6ca8a561783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:10.835778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 735/1472] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:09:10.924228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42800c21-b1ba-4a8f-b566-88b5c6d72f9d None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:10.924666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42800c21-b1ba-4a8f-b566-88b5c6d72f9d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:10.924954 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42800c21-b1ba-4a8f-b566-88b5c6d72f9d None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:10.926399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 738/1473] 10.222.0.26 () {54 vars in 962 bytes} [Mon Jul 4 09:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:10.933030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06e279-bde6-4fe6-98c7-ce7208390cff None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:10.933322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06e279-bde6-4fe6-98c7-ce7208390cff None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:10.933601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06e279-bde6-4fe6-98c7-ce7208390cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:10.952870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ee06e279-bde6-4fe6-98c7-ce7208390cff None None] MFA Rules not processed for user `107a90e384504e8d80264d6bf20f1d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:10.954112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ee06e279-bde6-4fe6-98c7-ce7208390cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:10.961816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 736/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Jul 4 09:09:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:09:10.969151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42800c21-b1ba-4a8f-b566-88b5c6d72f9d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:10.981487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42800c21-b1ba-4a8f-b566-88b5c6d72f9d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:10.985769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915b131-13d0-40cd-b18e-560b611a3e56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a90e384504e8d80264d6bf20f1d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:10.986434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915b131-13d0-40cd-b18e-560b611a3e56 service glance] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:10.986759 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915b131-13d0-40cd-b18e-560b611a3e56 service glance] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:10.987008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915b131-13d0-40cd-b18e-560b611a3e56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:10.996453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915b131-13d0-40cd-b18e-560b611a3e56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:10.997904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915b131-13d0-40cd-b18e-560b611a3e56 service glance] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:11.007462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 739/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:09:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:09:13.127039 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dccccf-5dbb-421f-9fd7-5b0f3ab1ae64 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:13.127647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dccccf-5dbb-421f-9fd7-5b0f3ab1ae64 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:13.127647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dccccf-5dbb-421f-9fd7-5b0f3ab1ae64 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:13.129105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 737/1476] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:13.136050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e8392-8ce0-4b8e-a561-4faf9e863119 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:13.136540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e8392-8ce0-4b8e-a561-4faf9e863119 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:13.136645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e8392-8ce0-4b8e-a561-4faf9e863119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:13.158443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-003e8392-8ce0-4b8e-a561-4faf9e863119 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:13.160216 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-003e8392-8ce0-4b8e-a561-4faf9e863119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:13.169040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 740/1477] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:09:13.193849 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dccccf-5dbb-421f-9fd7-5b0f3ab1ae64 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:13.216034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dccccf-5dbb-421f-9fd7-5b0f3ab1ae64 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:13.223534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c932fc-0987-4e70-b83e-45bc76458dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:13.224240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c932fc-0987-4e70-b83e-45bc76458dca admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:13.224527 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c932fc-0987-4e70-b83e-45bc76458dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:13.224803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c932fc-0987-4e70-b83e-45bc76458dca admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:13.225516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c932fc-0987-4e70-b83e-45bc76458dca admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:13.227406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-80c932fc-0987-4e70-b83e-45bc76458dca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:09:13.227406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 04 09:09:13.227406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: Jul 04 09:09:13.228548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c932fc-0987-4e70-b83e-45bc76458dca admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:13.248430 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 738/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 4 09:09:13 2022] GET /identity/v3/projects => generated 2553 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:09:15.294482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ddd01-9602-4968-99f3-851133ca658f None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:15.295074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ddd01-9602-4968-99f3-851133ca658f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:15.295074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ddd01-9602-4968-99f3-851133ca658f None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:15.297131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 741/1479] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:09:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:15.303224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac94d4a-5992-40f7-9ec9-d525b0f102d0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:15.303697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac94d4a-5992-40f7-9ec9-d525b0f102d0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:15.303803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac94d4a-5992-40f7-9ec9-d525b0f102d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:15.326431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7ac94d4a-5992-40f7-9ec9-d525b0f102d0 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:15.327646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7ac94d4a-5992-40f7-9ec9-d525b0f102d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:15.337086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 739/1480] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:09:15.380510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ddd01-9602-4968-99f3-851133ca658f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:15.397695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ddd01-9602-4968-99f3-851133ca658f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:15.403494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c842a4-cd37-45af-bf4a-8e96aaff2be2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:15.404128 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c842a4-cd37-45af-bf4a-8e96aaff2be2 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:15.404409 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c842a4-cd37-45af-bf4a-8e96aaff2be2 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:15.404680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c842a4-cd37-45af-bf4a-8e96aaff2be2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:15.415289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c842a4-cd37-45af-bf4a-8e96aaff2be2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:15.417609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c842a4-cd37-45af-bf4a-8e96aaff2be2 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:15.433228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 742/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:09:17.653636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7de5-781d-4591-97eb-28237cd4576c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:17.654651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7de5-781d-4591-97eb-28237cd4576c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:17.654651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7de5-781d-4591-97eb-28237cd4576c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:17.656927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 740/1482] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:09:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:17.663038 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbf34d-7a2b-412d-967e-ca814349805b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:17.663472 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbf34d-7a2b-412d-967e-ca814349805b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:17.663653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbf34d-7a2b-412d-967e-ca814349805b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:17.683762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1acbf34d-7a2b-412d-967e-ca814349805b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:17.684966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1acbf34d-7a2b-412d-967e-ca814349805b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:17.694600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 743/1483] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:09:17.740029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7de5-781d-4591-97eb-28237cd4576c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:17.755118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7de5-781d-4591-97eb-28237cd4576c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:17.759289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da500f50-1f0e-4a67-b08b-0ec850b721d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:17.760280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da500f50-1f0e-4a67-b08b-0ec850b721d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:17.760560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da500f50-1f0e-4a67-b08b-0ec850b721d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:17.760873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da500f50-1f0e-4a67-b08b-0ec850b721d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:17.772852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da500f50-1f0e-4a67-b08b-0ec850b721d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:17.774344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da500f50-1f0e-4a67-b08b-0ec850b721d5 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:17.784855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 741/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:09:19.973988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f2a3f-eab7-4bdf-94b9-166a5df0d1c4 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:19.974554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f2a3f-eab7-4bdf-94b9-166a5df0d1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:19.974554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f2a3f-eab7-4bdf-94b9-166a5df0d1c4 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:19.975842 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 744/1485] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:19.982347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd2622-9f17-4ff7-8a45-33103d2b2178 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:19.983105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd2622-9f17-4ff7-8a45-33103d2b2178 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:19.983301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd2622-9f17-4ff7-8a45-33103d2b2178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:20.010568 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-aecd2622-9f17-4ff7-8a45-33103d2b2178 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:20.011953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aecd2622-9f17-4ff7-8a45-33103d2b2178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:20.021876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 742/1486] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:09:20.060254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f2a3f-eab7-4bdf-94b9-166a5df0d1c4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:20.072769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f2a3f-eab7-4bdf-94b9-166a5df0d1c4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:20.076688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae83fa2-ab5a-48cd-832c-f137f98307f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:20.077324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae83fa2-ab5a-48cd-832c-f137f98307f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:20.077605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae83fa2-ab5a-48cd-832c-f137f98307f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:20.077904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae83fa2-ab5a-48cd-832c-f137f98307f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:20.087921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae83fa2-ab5a-48cd-832c-f137f98307f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:20.089369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae83fa2-ab5a-48cd-832c-f137f98307f1 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:20.100231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 745/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:09:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:09:22.252449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2540e05-daa9-459e-a473-1c665b898c0c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:22.253074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2540e05-daa9-459e-a473-1c665b898c0c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:22.253074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2540e05-daa9-459e-a473-1c665b898c0c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:22.254433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 743/1488] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:22.261282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89665614-bcf8-4f5b-b82d-ae7c78d87362 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:22.262087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89665614-bcf8-4f5b-b82d-ae7c78d87362 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:22.262212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89665614-bcf8-4f5b-b82d-ae7c78d87362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:22.288107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-89665614-bcf8-4f5b-b82d-ae7c78d87362 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:22.289314 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-89665614-bcf8-4f5b-b82d-ae7c78d87362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:22.297555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 746/1489] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:09:22.339457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2540e05-daa9-459e-a473-1c665b898c0c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:22.351025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2540e05-daa9-459e-a473-1c665b898c0c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:22.355411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbabc4-730f-4026-9c05-f598daac6f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:22.356027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbabc4-730f-4026-9c05-f598daac6f19 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:22.356319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbabc4-730f-4026-9c05-f598daac6f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:22.356596 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbabc4-730f-4026-9c05-f598daac6f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:22.366340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbabc4-730f-4026-9c05-f598daac6f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:22.367761 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbabc4-730f-4026-9c05-f598daac6f19 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:22.377672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 744/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:09:24.591824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872805b2-1dba-41e8-8cb3-d55f6844d17e None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:24.592382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872805b2-1dba-41e8-8cb3-d55f6844d17e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:24.592382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872805b2-1dba-41e8-8cb3-d55f6844d17e None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:24.593838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 747/1491] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 4 09:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:09:24.599411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248db0b-d0f9-4800-917d-07bfb10156a1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:24.599880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248db0b-d0f9-4800-917d-07bfb10156a1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:24.599984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248db0b-d0f9-4800-917d-07bfb10156a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:24.620371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0248db0b-d0f9-4800-917d-07bfb10156a1 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:09:24.621573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0248db0b-d0f9-4800-917d-07bfb10156a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:09:24.630318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 745/1492] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 4 09:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:09:24.671735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872805b2-1dba-41e8-8cb3-d55f6844d17e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:09:24.684532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872805b2-1dba-41e8-8cb3-d55f6844d17e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:09:24.688756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fca78c-40c6-4a03-b2a2-edb50cca48f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:09:24.689516 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fca78c-40c6-4a03-b2a2-edb50cca48f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:09:24.689802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fca78c-40c6-4a03-b2a2-edb50cca48f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:09:24.690116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fca78c-40c6-4a03-b2a2-edb50cca48f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:09:24.699404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fca78c-40c6-4a03-b2a2-edb50cca48f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:09:24.700842 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fca78c-40c6-4a03-b2a2-edb50cca48f5 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:09:24.710414 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 748/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:09:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:09:34.613927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=33 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:09:34.706274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=33 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:10:58.306520 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b000e-03ca-4d8e-98e1-ca45856a5e17 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.307480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b000e-03ca-4d8e-98e1-ca45856a5e17 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.307480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b000e-03ca-4d8e-98e1-ca45856a5e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.332000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-271b000e-03ca-4d8e-98e1-ca45856a5e17 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:10:58.334028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-271b000e-03ca-4d8e-98e1-ca45856a5e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:10:58.346208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 746/1494] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:10:58.354242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586688-6ea0-4d6b-832a-53b2e4b9c4ae None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.354773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586688-6ea0-4d6b-832a-53b2e4b9c4ae None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.354773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586688-6ea0-4d6b-832a-53b2e4b9c4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.374362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ae586688-6ea0-4d6b-832a-53b2e4b9c4ae None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:10:58.375550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ae586688-6ea0-4d6b-832a-53b2e4b9c4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:10:58.385493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 749/1495] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:10:58.392057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b000e-03ca-4d8e-98e1-ca45856a5e17 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:58.402327 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b000e-03ca-4d8e-98e1-ca45856a5e17 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:58.408198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13963b76-0632-4b60-8765-fecf5a9445e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:58.408791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13963b76-0632-4b60-8765-fecf5a9445e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.409072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13963b76-0632-4b60-8765-fecf5a9445e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.409344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13963b76-0632-4b60-8765-fecf5a9445e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.410112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13963b76-0632-4b60-8765-fecf5a9445e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:10:58.410416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13963b76-0632-4b60-8765-fecf5a9445e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:58.411432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-13963b76-0632-4b60-8765-fecf5a9445e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:10:58.411432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 04 09:10:58.411432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: Jul 04 09:10:58.412328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13963b76-0632-4b60-8765-fecf5a9445e0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:58.424645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 747/1496] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:10:58.432712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae586688-6ea0-4d6b-832a-53b2e4b9c4ae None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:58.442110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae586688-6ea0-4d6b-832a-53b2e4b9c4ae None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:58.450182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef6ba0-7553-4588-a1bc-4d9336cd3c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:58.451126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef6ba0-7553-4588-a1bc-4d9336cd3c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.451410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef6ba0-7553-4588-a1bc-4d9336cd3c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.451682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef6ba0-7553-4588-a1bc-4d9336cd3c9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.452370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef6ba0-7553-4588-a1bc-4d9336cd3c9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:58.454249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-a2ef6ba0-7553-4588-a1bc-4d9336cd3c9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:10:58.454249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jul 04 09:10:58.454249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: Jul 04 09:10:58.455445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef6ba0-7553-4588-a1bc-4d9336cd3c9e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:58.509604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 750/1497] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:10:58 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:10:58.514477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13963b76-0632-4b60-8765-fecf5a9445e0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:58.525252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13963b76-0632-4b60-8765-fecf5a9445e0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:58.593134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c5ab1-1ca4-4c52-b672-2eebdab1e8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:58.593834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c5ab1-1ca4-4c52-b672-2eebdab1e8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.594149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c5ab1-1ca4-4c52-b672-2eebdab1e8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.594426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c5ab1-1ca4-4c52-b672-2eebdab1e8a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.595115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c5ab1-1ca4-4c52-b672-2eebdab1e8a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:58.596170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-380c5ab1-1ca4-4c52-b672-2eebdab1e8a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:10:58.596170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 04 09:10:58.596170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: Jul 04 09:10:58.597394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c5ab1-1ca4-4c52-b672-2eebdab1e8a1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:58.618055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 748/1498] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:10:58 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:10:58.622328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef6ba0-7553-4588-a1bc-4d9336cd3c9e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:58.633466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef6ba0-7553-4588-a1bc-4d9336cd3c9e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:58.638878 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b071ea-fe66-4150-bbe1-4a094b809529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:58.639539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b071ea-fe66-4150-bbe1-4a094b809529 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.639823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b071ea-fe66-4150-bbe1-4a094b809529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.640098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b071ea-fe66-4150-bbe1-4a094b809529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.640820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b071ea-fe66-4150-bbe1-4a094b809529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:58.641830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-24b071ea-fe66-4150-bbe1-4a094b809529 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:10:58.641830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jul 04 09:10:58.641830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: Jul 04 09:10:58.642758 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b071ea-fe66-4150-bbe1-4a094b809529 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:58.655810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 751/1499] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:10:58.659891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c5ab1-1ca4-4c52-b672-2eebdab1e8a1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:58.670713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c5ab1-1ca4-4c52-b672-2eebdab1e8a1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:58.675943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795af59a-1c4e-4ff8-8ca7-005966b8e05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:58.676582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795af59a-1c4e-4ff8-8ca7-005966b8e05d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.676881 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795af59a-1c4e-4ff8-8ca7-005966b8e05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.677156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795af59a-1c4e-4ff8-8ca7-005966b8e05d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.677910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795af59a-1c4e-4ff8-8ca7-005966b8e05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:58.678930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-795af59a-1c4e-4ff8-8ca7-005966b8e05d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:10:58.678930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 04 09:10:58.678930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: Jul 04 09:10:58.679808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795af59a-1c4e-4ff8-8ca7-005966b8e05d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:58.693261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 749/1500] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:10:58.697519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b071ea-fe66-4150-bbe1-4a094b809529 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:58.708088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b071ea-fe66-4150-bbe1-4a094b809529 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:58.712949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af4517-a781-4716-97fc-fa34e8b2528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:58.713829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4517-a781-4716-97fc-fa34e8b2528e admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.714141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4517-a781-4716-97fc-fa34e8b2528e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.714416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4517-a781-4716-97fc-fa34e8b2528e admin admin] PATH_INFO: `/v3/projects/cebd8e8b85f7444a9dbbb07a1dd335ac/users/bba25d6751fc4a41a0c7916d097f9e6e/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.731407 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af4517-a781-4716-97fc-fa34e8b2528e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebd8e8b85f7444a9dbbb07a1dd335ac, user_id=bba25d6751fc4a41a0c7916d097f9e6e, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:58.732598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-58af4517-a781-4716-97fc-fa34e8b2528e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:10:58.732598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jul 04 09:10:58.732598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: Jul 04 09:10:58.734581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af4517-a781-4716-97fc-fa34e8b2528e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:58.751105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 752/1501] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:10:58 2022] PUT /identity/v3/projects/cebd8e8b85f7444a9dbbb07a1dd335ac/users/bba25d6751fc4a41a0c7916d097f9e6e/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:10:58.763495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d01478-9e14-40d3-bae5-79bb56d2e442 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.763791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d01478-9e14-40d3-bae5-79bb56d2e442 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.764071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d01478-9e14-40d3-bae5-79bb56d2e442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.792030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-79d01478-9e14-40d3-bae5-79bb56d2e442 None None] MFA Rules not processed for user `bba25d6751fc4a41a0c7916d097f9e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:10:58.793246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-79d01478-9e14-40d3-bae5-79bb56d2e442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:10:58.861546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 750/1502] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:10:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:10:58.874935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c752a-cd59-48ef-ae79-6eb1298a239d None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.875223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c752a-cd59-48ef-ae79-6eb1298a239d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.875499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c752a-cd59-48ef-ae79-6eb1298a239d None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.876864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 753/1503] 10.222.0.26 () {54 vars in 963 bytes} [Mon Jul 4 09:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:10:58.881921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f0634-0f50-4f0d-8fce-c39979513f0b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.882514 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f0634-0f50-4f0d-8fce-c39979513f0b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.882811 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f0634-0f50-4f0d-8fce-c39979513f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.901278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-082f0634-0f50-4f0d-8fce-c39979513f0b None None] MFA Rules not processed for user `e9f34c35a9ba4376a9b55ec5e23cf35f`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:10:58.902505 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-082f0634-0f50-4f0d-8fce-c39979513f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:10:58.949727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:10:58.958259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 751/1504] 10.222.0.26 () {58 vars in 1060 bytes} [Mon Jul 4 09:10:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:10:58.966581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c752a-cd59-48ef-ae79-6eb1298a239d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:58.978331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c752a-cd59-48ef-ae79-6eb1298a239d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:58.982011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59cae7-0abd-4410-ac94-8ea9df457790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:58.982725 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59cae7-0abd-4410-ac94-8ea9df457790 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:58.983005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59cae7-0abd-4410-ac94-8ea9df457790 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:58.983281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59cae7-0abd-4410-ac94-8ea9df457790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:58.992071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59cae7-0abd-4410-ac94-8ea9df457790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:58.993506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59cae7-0abd-4410-ac94-8ea9df457790 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:59.002664 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 754/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:10:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:10:59.048743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3954a3-4ea4-4022-90ae-a26db10b2be2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:59.049039 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3954a3-4ea4-4022-90ae-a26db10b2be2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:59.049322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3954a3-4ea4-4022-90ae-a26db10b2be2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:59.050694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 752/1506] 10.222.0.26 () {52 vars in 829 bytes} [Mon Jul 4 09:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:10:59.054489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59cae7-0abd-4410-ac94-8ea9df457790 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:59.118533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59cae7-0abd-4410-ac94-8ea9df457790 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:59.126413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:59.127294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:59.127572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:59.127843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] PATH_INFO: `/v3/users/bba25d6751fc4a41a0c7916d097f9e6e` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:59.129161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bba25d6751fc4a41a0c7916d097f9e6e)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:59.130220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:10:59.130220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jul 04 09:10:59.130220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: Jul 04 09:10:59.131460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:59.202085 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bba25d6751fc4a41a0c7916d097f9e6e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:10:59.202707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bba25d6751fc4a41a0c7916d097f9e6e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:10:59.213383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bba25d6751fc4a41a0c7916d097f9e6e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:10:59.225523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bba25d6751fc4a41a0c7916d097f9e6e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:10:59.226305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bba25d6751fc4a41a0c7916d097f9e6e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:10:59.245770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:10:59.255835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 755/1507] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:10:59 2022] DELETE /identity/v3/users/bba25d6751fc4a41a0c7916d097f9e6e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:10:59.260655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290328ab-275c-4c08-8588-c2f1e5ee9a3c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:59.260950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290328ab-275c-4c08-8588-c2f1e5ee9a3c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:59.261260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290328ab-275c-4c08-8588-c2f1e5ee9a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:59.279585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-290328ab-275c-4c08-8588-c2f1e5ee9a3c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:10:59.280812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-290328ab-275c-4c08-8588-c2f1e5ee9a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:10:59.341759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 753/1508] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:10:59.355066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:59.357213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:10:59.417322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094ae7d-e7dc-4bd8-b97f-0410015b04bd admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:59.421531 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d996d86-8e72-4459-b751-3b26d1cf3e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:59.422210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d996d86-8e72-4459-b751-3b26d1cf3e45 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:59.422490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d996d86-8e72-4459-b751-3b26d1cf3e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:59.422763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d996d86-8e72-4459-b751-3b26d1cf3e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:59.432332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d996d86-8e72-4459-b751-3b26d1cf3e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:59.433774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d996d86-8e72-4459-b751-3b26d1cf3e45 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:59.443452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 756/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:10:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:10:59.888093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290328ab-275c-4c08-8588-c2f1e5ee9a3c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:10:59.889805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-290328ab-275c-4c08-8588-c2f1e5ee9a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:10:59.902052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290328ab-275c-4c08-8588-c2f1e5ee9a3c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:10:59.907732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:10:59.908641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:10:59.908927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:10:59.909204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] PATH_INFO: `/v3/projects/cebd8e8b85f7444a9dbbb07a1dd335ac` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:10:59.910516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cebd8e8b85f7444a9dbbb07a1dd335ac)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:10:59.911554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:10:59.911554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 04 09:10:59.911554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: Jul 04 09:10:59.912891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:10:59.975800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cebd8e8b85f7444a9dbbb07a1dd335ac', 'request_id': 'req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:10:59.986604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cebd8e8b85f7444a9dbbb07a1dd335ac', 'request_id': 'req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:11:00.005007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cebd8e8b85f7444a9dbbb07a1dd335ac', 'request_id': 'req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:11:00.020660 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] The token cache is being invalidate because project cebd8e8b85f7444a9dbbb07a1dd335ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:11:00.020954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:11:00.021719 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aabe5fde-05eb-4dc8-acef-ede9e5547bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:11:00.023472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 754/1510] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:10:59 2022] DELETE /identity/v3/projects/cebd8e8b85f7444a9dbbb07a1dd335ac => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:11:09.439231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:11:10.016950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:12:43.948462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa702e0-c08d-477b-abeb-a9a2faa0a210 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:43.949217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa702e0-c08d-477b-abeb-a9a2faa0a210 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:43.949217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa702e0-c08d-477b-abeb-a9a2faa0a210 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:43.950744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:43.956360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3e56-70b1-4e33-a1b7-df584b4bd25d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:43.956827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3e56-70b1-4e33-a1b7-df584b4bd25d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:43.956960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3e56-70b1-4e33-a1b7-df584b4bd25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:43.977757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-aaff3e56-70b1-4e33-a1b7-df584b4bd25d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:43.979014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aaff3e56-70b1-4e33-a1b7-df584b4bd25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:44.043693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:44.215286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa702e0-c08d-477b-abeb-a9a2faa0a210 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:44.218479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4fa702e0-c08d-477b-abeb-a9a2faa0a210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:44.303314 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa702e0-c08d-477b-abeb-a9a2faa0a210 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:44.309537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdedc58-5bb1-4f2c-95c1-cd1042e4bd46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:44.310295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdedc58-5bb1-4f2c-95c1-cd1042e4bd46 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:44.310581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdedc58-5bb1-4f2c-95c1-cd1042e4bd46 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:44.310855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdedc58-5bb1-4f2c-95c1-cd1042e4bd46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:44.319674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdedc58-5bb1-4f2c-95c1-cd1042e4bd46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:44.321279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdedc58-5bb1-4f2c-95c1-cd1042e4bd46 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:44.331773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 758/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:45.170052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d82cb2-037a-4309-8f3a-eceda90fa233 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:45.170530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d82cb2-037a-4309-8f3a-eceda90fa233 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:45.170632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d82cb2-037a-4309-8f3a-eceda90fa233 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:45.172690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:45.179572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc559e-d60a-42d0-b1c8-21d01d7140e2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:45.180062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc559e-d60a-42d0-b1c8-21d01d7140e2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:45.180163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc559e-d60a-42d0-b1c8-21d01d7140e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:45.200924 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f2dc559e-d60a-42d0-b1c8-21d01d7140e2 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:45.202274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f2dc559e-d60a-42d0-b1c8-21d01d7140e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:45.211614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:45.368139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d82cb2-037a-4309-8f3a-eceda90fa233 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:45.385186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d82cb2-037a-4309-8f3a-eceda90fa233 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:45.391084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f791fd-171a-482f-96b9-dd470b4a6448 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:45.391724 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f791fd-171a-482f-96b9-dd470b4a6448 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:45.392005 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f791fd-171a-482f-96b9-dd470b4a6448 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:45.392279 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f791fd-171a-482f-96b9-dd470b4a6448 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:45.401210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f791fd-171a-482f-96b9-dd470b4a6448 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:45.402928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f791fd-171a-482f-96b9-dd470b4a6448 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:45.412939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 757/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:45.907930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4a22e-faf4-4371-b2b9-03a56ece5cf4 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:45.908196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4a22e-faf4-4371-b2b9-03a56ece5cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:45.908492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4a22e-faf4-4371-b2b9-03a56ece5cf4 None None] PATH_INFO: `/v3` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:45.909819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 760/1517] 10.222.0.57 () {54 vars in 925 bytes} [Mon Jul 4 09:12:45 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:12:45.932354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93eb6c-e47b-454d-9776-77f4debe626d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:45.932649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93eb6c-e47b-454d-9776-77f4debe626d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:45.933068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93eb6c-e47b-454d-9776-77f4debe626d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:45.957360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-dc93eb6c-e47b-454d-9776-77f4debe626d None None] MFA Rules not processed for user `aff2305249a242639e1415094bf9df33`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:45.958610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dc93eb6c-e47b-454d-9776-77f4debe626d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:46.031545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 758/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Jul 4 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:49.812622 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875a149-d4b5-41ca-ad09-2a385c3611cd None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:49.813170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875a149-d4b5-41ca-ad09-2a385c3611cd None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:49.813170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875a149-d4b5-41ca-ad09-2a385c3611cd None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:49.814481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 761/1519] 10.222.0.26 () {54 vars in 960 bytes} [Mon Jul 4 09:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:49.819639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c35b1-f8c4-46e3-800a-a6a6c353c94d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:49.820105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c35b1-f8c4-46e3-800a-a6a6c353c94d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:49.820242 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c35b1-f8c4-46e3-800a-a6a6c353c94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:49.840480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bd5c35b1-f8c4-46e3-800a-a6a6c353c94d None None] MFA Rules not processed for user `aff2305249a242639e1415094bf9df33`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:49.841698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bd5c35b1-f8c4-46e3-800a-a6a6c353c94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:49.850613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 759/1520] 10.222.0.26 () {58 vars in 1057 bytes} [Mon Jul 4 09:12:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:12:49.858904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875a149-d4b5-41ca-ad09-2a385c3611cd None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:49.874412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875a149-d4b5-41ca-ad09-2a385c3611cd None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:49.880116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f03647-a5f8-4a82-bfb3-2e7641b5c22c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:49.880877 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f03647-a5f8-4a82-bfb3-2e7641b5c22c service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:49.881168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f03647-a5f8-4a82-bfb3-2e7641b5c22c service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:49.881506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f03647-a5f8-4a82-bfb3-2e7641b5c22c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:49.891185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f03647-a5f8-4a82-bfb3-2e7641b5c22c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:49.892781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f03647-a5f8-4a82-bfb3-2e7641b5c22c service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:49.902992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 762/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:56.676866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3ea06-36af-4120-b6ea-bd74d91e0ca9 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:56.677469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3ea06-36af-4120-b6ea-bd74d91e0ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:56.677469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3ea06-36af-4120-b6ea-bd74d91e0ca9 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:56.678857 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:56.684969 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc1dd2-7fb9-42e7-ad8c-882e44818d10 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:56.685754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc1dd2-7fb9-42e7-ad8c-882e44818d10 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:56.685933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc1dd2-7fb9-42e7-ad8c-882e44818d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:56.709857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-56fc1dd2-7fb9-42e7-ad8c-882e44818d10 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:56.711155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-56fc1dd2-7fb9-42e7-ad8c-882e44818d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:56.719598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:56.875752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3ea06-36af-4120-b6ea-bd74d91e0ca9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:56.891561 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3ea06-36af-4120-b6ea-bd74d91e0ca9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:56.897444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ab479-9f7e-4987-8238-9323e85ad81c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:56.898118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ab479-9f7e-4987-8238-9323e85ad81c service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:56.898401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ab479-9f7e-4987-8238-9323e85ad81c service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:56.898680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ab479-9f7e-4987-8238-9323e85ad81c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:56.907144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ab479-9f7e-4987-8238-9323e85ad81c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:56.908742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ab479-9f7e-4987-8238-9323e85ad81c service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:56.918392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 761/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:12:57.728606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d710c-6574-49da-9576-441d3736f9c7 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:57.732028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d710c-6574-49da-9576-441d3736f9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:57.732028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d710c-6574-49da-9576-441d3736f9c7 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:57.732028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 4 09:12:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:12:57.737728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd46498-5f22-4990-8aa8-87dd62fd44c4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:57.738507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd46498-5f22-4990-8aa8-87dd62fd44c4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:57.738676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd46498-5f22-4990-8aa8-87dd62fd44c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:57.761276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-efd46498-5f22-4990-8aa8-87dd62fd44c4 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:12:57.762532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-efd46498-5f22-4990-8aa8-87dd62fd44c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:12:57.772706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 4 09:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:12:57.927603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d710c-6574-49da-9576-441d3736f9c7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:12:57.945154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d710c-6574-49da-9576-441d3736f9c7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:12:57.951887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a09cf-68ec-4736-9f8c-c881bf89429c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:12:57.952576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a09cf-68ec-4736-9f8c-c881bf89429c service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:12:57.952819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a09cf-68ec-4736-9f8c-c881bf89429c service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:12:57.953124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a09cf-68ec-4736-9f8c-c881bf89429c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:12:57.961773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a09cf-68ec-4736-9f8c-c881bf89429c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:12:57.963488 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a09cf-68ec-4736-9f8c-c881bf89429c service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:12:57.973737 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 765/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:12:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:01.786679 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32f1e7-83d0-4e29-b6a4-dad03f011910 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:01.787375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32f1e7-83d0-4e29-b6a4-dad03f011910 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:01.787375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32f1e7-83d0-4e29-b6a4-dad03f011910 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:01.788617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:01.795405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1762401-7311-4b6d-a21d-982019702081 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:01.796184 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1762401-7311-4b6d-a21d-982019702081 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:01.796295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1762401-7311-4b6d-a21d-982019702081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:01.824863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a1762401-7311-4b6d-a21d-982019702081 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:01.826106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a1762401-7311-4b6d-a21d-982019702081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:01.836051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:01.912860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32f1e7-83d0-4e29-b6a4-dad03f011910 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:01.916065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ca32f1e7-83d0-4e29-b6a4-dad03f011910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:01.985229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32f1e7-83d0-4e29-b6a4-dad03f011910 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:01.989296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78b62f-2838-457e-8ba6-472ad15a2345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:01.989942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b62f-2838-457e-8ba6-472ad15a2345 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:01.990255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b62f-2838-457e-8ba6-472ad15a2345 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:01.990531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b62f-2838-457e-8ba6-472ad15a2345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:02.000023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78b62f-2838-457e-8ba6-472ad15a2345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:02.001477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78b62f-2838-457e-8ba6-472ad15a2345 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:02.006433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:13:02.011896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 764/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:04.038550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a205d3b-acbc-4434-9312-d80759d7c629 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:04.039106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a205d3b-acbc-4434-9312-d80759d7c629 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:04.039106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a205d3b-acbc-4434-9312-d80759d7c629 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:04.041320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:04.047894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d86cce-efba-43af-b5a5-c2c69c44b0ec None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:04.048412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d86cce-efba-43af-b5a5-c2c69c44b0ec None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:04.048519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d86cce-efba-43af-b5a5-c2c69c44b0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:04.075821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-54d86cce-efba-43af-b5a5-c2c69c44b0ec None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:04.077073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-54d86cce-efba-43af-b5a5-c2c69c44b0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:04.085518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:04.125232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a205d3b-acbc-4434-9312-d80759d7c629 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:04.139234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a205d3b-acbc-4434-9312-d80759d7c629 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:04.143161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11591805-dcf6-4a95-abc1-e6021c69140a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:04.143812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591805-dcf6-4a95-abc1-e6021c69140a service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:04.144096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591805-dcf6-4a95-abc1-e6021c69140a service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:04.144374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591805-dcf6-4a95-abc1-e6021c69140a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:04.153767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11591805-dcf6-4a95-abc1-e6021c69140a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:04.155236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11591805-dcf6-4a95-abc1-e6021c69140a service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:04.164824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 768/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:07.629665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c5b43-5773-43ff-bfb5-13b8451fc106 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:07.630648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c5b43-5773-43ff-bfb5-13b8451fc106 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:07.630648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c5b43-5773-43ff-bfb5-13b8451fc106 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:07.633285 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:07.639816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e738-9c64-4eca-b2dd-477346b83842 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:07.640607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e738-9c64-4eca-b2dd-477346b83842 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:07.640755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e738-9c64-4eca-b2dd-477346b83842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:07.663806 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f6d5e738-9c64-4eca-b2dd-477346b83842 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:07.664963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f6d5e738-9c64-4eca-b2dd-477346b83842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:07.673537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:07.983299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c5b43-5773-43ff-bfb5-13b8451fc106 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:07.985628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a82c5b43-5773-43ff-bfb5-13b8451fc106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:08.048260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c5b43-5773-43ff-bfb5-13b8451fc106 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:08.051843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33757e87-3e6e-49d4-9e7d-0e25997c2073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:08.052776 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33757e87-3e6e-49d4-9e7d-0e25997c2073 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:08.053070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33757e87-3e6e-49d4-9e7d-0e25997c2073 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:08.053348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33757e87-3e6e-49d4-9e7d-0e25997c2073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:08.062381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33757e87-3e6e-49d4-9e7d-0e25997c2073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:08.063820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33757e87-3e6e-49d4-9e7d-0e25997c2073 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:08.072887 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 767/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:10.191326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545be3e6-89df-45d0-a62d-cab5561fe10b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:10.191893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545be3e6-89df-45d0-a62d-cab5561fe10b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:10.192030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545be3e6-89df-45d0-a62d-cab5561fe10b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:10.193532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 4 09:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:13:10.199684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd86305-f157-4d2c-bc56-f2c0e2d96aff None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:10.200414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd86305-f157-4d2c-bc56-f2c0e2d96aff None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:10.200530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd86305-f157-4d2c-bc56-f2c0e2d96aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:10.230527 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2bd86305-f157-4d2c-bc56-f2c0e2d96aff None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:13:10.231708 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2bd86305-f157-4d2c-bc56-f2c0e2d96aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:10.242059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 4 09:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:13:10.560640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545be3e6-89df-45d0-a62d-cab5561fe10b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:13:10.563028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-545be3e6-89df-45d0-a62d-cab5561fe10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:13:10.571554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=65 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:13:10.581730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545be3e6-89df-45d0-a62d-cab5561fe10b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:13:10.588905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424801b-5633-43a6-8442-fdbec054089e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:13:10.589585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424801b-5633-43a6-8442-fdbec054089e service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:13:10.589864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424801b-5633-43a6-8442-fdbec054089e service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:13:10.590188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424801b-5633-43a6-8442-fdbec054089e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:13:10.601101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424801b-5633-43a6-8442-fdbec054089e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:13:10.602630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424801b-5633-43a6-8442-fdbec054089e service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:13:10.612614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 771/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:13:20.221227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=32 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:13:20.608237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:40.452581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5e09d-3057-4e17-8817-e714346e7b17 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:40.453753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5e09d-3057-4e17-8817-e714346e7b17 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:40.454269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5e09d-3057-4e17-8817-e714346e7b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:40.479202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-57a5e09d-3057-4e17-8817-e714346e7b17 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:40.480799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-57a5e09d-3057-4e17-8817-e714346e7b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:40.493825 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 769/1540] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:40.502412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0d06f-ab2c-4360-9fc4-8ff525a9c823 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:40.503419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0d06f-ab2c-4360-9fc4-8ff525a9c823 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:40.503915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0d06f-ab2c-4360-9fc4-8ff525a9c823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:40.531040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b3a0d06f-ab2c-4360-9fc4-8ff525a9c823 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:40.532926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b3a0d06f-ab2c-4360-9fc4-8ff525a9c823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:40.547268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 772/1541] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:40.553064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5e09d-3057-4e17-8817-e714346e7b17 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:40.568415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5e09d-3057-4e17-8817-e714346e7b17 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:40.577379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48200805-5765-4058-968a-523275fcc3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:40.578112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48200805-5765-4058-968a-523275fcc3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:40.578399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48200805-5765-4058-968a-523275fcc3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:40.578672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48200805-5765-4058-968a-523275fcc3a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:40.579434 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48200805-5765-4058-968a-523275fcc3a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:14:40.579742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48200805-5765-4058-968a-523275fcc3a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:40.581417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48200805-5765-4058-968a-523275fcc3a4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:40.603864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 770/1542] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:14:40.612727 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0d06f-ab2c-4360-9fc4-8ff525a9c823 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:40.623448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0d06f-ab2c-4360-9fc4-8ff525a9c823 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:40.634155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c08a6-e5ee-4b93-a255-12eb98c2c31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:40.635288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c08a6-e5ee-4b93-a255-12eb98c2c31e admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:40.635820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c08a6-e5ee-4b93-a255-12eb98c2c31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:40.636353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c08a6-e5ee-4b93-a255-12eb98c2c31e admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:40.637295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c08a6-e5ee-4b93-a255-12eb98c2c31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:40.639508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c08a6-e5ee-4b93-a255-12eb98c2c31e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:40.698640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 773/1543] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:40 2022] POST /identity/v3/projects => generated 357 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:40.703451 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48200805-5765-4058-968a-523275fcc3a4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:40.720439 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48200805-5765-4058-968a-523275fcc3a4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:40.815074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b9eac-46f8-44a4-aa1f-0a5c6e4b676d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:40.815763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b9eac-46f8-44a4-aa1f-0a5c6e4b676d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:40.816048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b9eac-46f8-44a4-aa1f-0a5c6e4b676d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:40.816320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b9eac-46f8-44a4-aa1f-0a5c6e4b676d admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:40.817020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b9eac-46f8-44a4-aa1f-0a5c6e4b676d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:40.819017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b9eac-46f8-44a4-aa1f-0a5c6e4b676d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:40.857601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e316e-3513-4a96-a022-b46881e3d482 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:40.858296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e316e-3513-4a96-a022-b46881e3d482 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:40.858802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e316e-3513-4a96-a022-b46881e3d482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:40.884248 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 771/1544] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:40 2022] POST /identity/v3/users => generated 279 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:40.892336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dceed3-0788-4618-8821-b788e341e01a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:40.892873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dceed3-0788-4618-8821-b788e341e01a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:40.893393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dceed3-0788-4618-8821-b788e341e01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:40.895629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-942e316e-3513-4a96-a022-b46881e3d482 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:40.897598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-942e316e-3513-4a96-a022-b46881e3d482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:40.916497 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 774/1545] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:14:40.917983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e316e-3513-4a96-a022-b46881e3d482 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:40.935817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-17dceed3-0788-4618-8821-b788e341e01a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:40.937432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-17dceed3-0788-4618-8821-b788e341e01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:40.946892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e316e-3513-4a96-a022-b46881e3d482 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:40.955277 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 772/1546] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:40.956120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056c04d-226c-417a-a1d3-b3f492092bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:40.956539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c0662-ef45-443e-bb7d-d3646341acbf None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:40.956829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c0662-ef45-443e-bb7d-d3646341acbf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:40.957108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c0662-ef45-443e-bb7d-d3646341acbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:40.957774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056c04d-226c-417a-a1d3-b3f492092bea admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:40.958297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056c04d-226c-417a-a1d3-b3f492092bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:40.958789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056c04d-226c-417a-a1d3-b3f492092bea admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:40.959746 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056c04d-226c-417a-a1d3-b3f492092bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:40.963729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056c04d-226c-417a-a1d3-b3f492092bea admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:40.984349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 775/1547] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:40.986092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ecd86-9633-490d-9694-577abd5fd3f8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:40.986457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-356c0662-ef45-443e-bb7d-d3646341acbf None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:40.986845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ecd86-9633-490d-9694-577abd5fd3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:40.987323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ecd86-9633-490d-9694-577abd5fd3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:40.987736 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-356c0662-ef45-443e-bb7d-d3646341acbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.010344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4e6ecd86-9633-490d-9694-577abd5fd3f8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.012085 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4e6ecd86-9633-490d-9694-577abd5fd3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.012477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 773/1548] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.017789 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629d359-0472-46e8-9c7d-a62666b9445b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.017789 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629d359-0472-46e8-9c7d-a62666b9445b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.017789 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629d359-0472-46e8-9c7d-a62666b9445b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.035829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 776/1549] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.037375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d665bf-7657-4b16-9a4f-9b3d54ae5e80 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.037670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d665bf-7657-4b16-9a4f-9b3d54ae5e80 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.038000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d665bf-7657-4b16-9a4f-9b3d54ae5e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.050772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c629d359-0472-46e8-9c7d-a62666b9445b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.053098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c629d359-0472-46e8-9c7d-a62666b9445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.069391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 774/1550] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.070398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629d359-0472-46e8-9c7d-a62666b9445b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.075913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-19d665bf-7657-4b16-9a4f-9b3d54ae5e80 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.077501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-19d665bf-7657-4b16-9a4f-9b3d54ae5e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.081500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629d359-0472-46e8-9c7d-a62666b9445b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.086335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89672ce-bfb5-4bb4-b0b3-0d94cc684c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.087015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89672ce-bfb5-4bb4-b0b3-0d94cc684c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.087304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89672ce-bfb5-4bb4-b0b3-0d94cc684c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.087585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89672ce-bfb5-4bb4-b0b3-0d94cc684c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.088323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89672ce-bfb5-4bb4-b0b3-0d94cc684c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.089213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 777/1551] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.092877 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d665bf-7657-4b16-9a4f-9b3d54ae5e80 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.093021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89672ce-bfb5-4bb4-b0b3-0d94cc684c5a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.102042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 775/1552] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:41.104653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9171-f691-4cb0-845c-3de4f96d8a77 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.105154 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9171-f691-4cb0-845c-3de4f96d8a77 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.105726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9171-f691-4cb0-845c-3de4f96d8a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.106999 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d665bf-7657-4b16-9a4f-9b3d54ae5e80 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.112387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8543d84-0623-4451-a81b-c0b529bbc15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.113169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8543d84-0623-4451-a81b-c0b529bbc15a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.113519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8543d84-0623-4451-a81b-c0b529bbc15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.113864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8543d84-0623-4451-a81b-c0b529bbc15a admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.114691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8543d84-0623-4451-a81b-c0b529bbc15a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:14:41.115063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8543d84-0623-4451-a81b-c0b529bbc15a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.116372 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-a8543d84-0623-4451-a81b-c0b529bbc15a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:14:41.116372 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jul 04 09:14:41.116372 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: Jul 04 09:14:41.117061 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8543d84-0623-4451-a81b-c0b529bbc15a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.133591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 778/1553] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:14:41.135036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0b8f3-d0f3-4f42-9e4e-b8106e9733d2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.135330 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0b8f3-d0f3-4f42-9e4e-b8106e9733d2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.135610 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0b8f3-d0f3-4f42-9e4e-b8106e9733d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.140251 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-40eb9171-f691-4cb0-845c-3de4f96d8a77 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.141789 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-40eb9171-f691-4cb0-845c-3de4f96d8a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.158073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b7e0b8f3-d0f3-4f42-9e4e-b8106e9733d2 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.159374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b7e0b8f3-d0f3-4f42-9e4e-b8106e9733d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.164970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 776/1554] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.164970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2f356-c6c0-467e-a2f7-15b98b42b228 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.165227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2f356-c6c0-467e-a2f7-15b98b42b228 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.165668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2f356-c6c0-467e-a2f7-15b98b42b228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.170178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 779/1555] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.171496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b1c13-44b6-4f30-9d06-f6655b46b942 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.171782 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b1c13-44b6-4f30-9d06-f6655b46b942 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.172051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b1c13-44b6-4f30-9d06-f6655b46b942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.187829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0be2f356-c6c0-467e-a2f7-15b98b42b228 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.189154 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0be2f356-c6c0-467e-a2f7-15b98b42b228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.191018 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-180b1c13-44b6-4f30-9d06-f6655b46b942 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.192205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-180b1c13-44b6-4f30-9d06-f6655b46b942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.197199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 777/1556] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.197960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2f356-c6c0-467e-a2f7-15b98b42b228 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.200401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 780/1557] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.201349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b1c13-44b6-4f30-9d06-f6655b46b942 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.211808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2f356-c6c0-467e-a2f7-15b98b42b228 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.211981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b1c13-44b6-4f30-9d06-f6655b46b942 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.216804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342fb02-73ed-4444-90a2-32fd19cad5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.216976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81466a54-554c-4bf4-a5e7-cc5356229f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.217464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342fb02-73ed-4444-90a2-32fd19cad5ce admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.217746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342fb02-73ed-4444-90a2-32fd19cad5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.217856 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466a54-554c-4bf4-a5e7-cc5356229f4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.218052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342fb02-73ed-4444-90a2-32fd19cad5ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.218161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466a54-554c-4bf4-a5e7-cc5356229f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.218397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81466a54-554c-4bf4-a5e7-cc5356229f4a admin admin] PATH_INFO: `/v3/projects/f04d65b2185b4f10a065ae5bf24245ec/users/13ae9af323874e0eb4807f02c9d6ed31/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.218786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342fb02-73ed-4444-90a2-32fd19cad5ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:14:41.219089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342fb02-73ed-4444-90a2-32fd19cad5ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.220685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342fb02-73ed-4444-90a2-32fd19cad5ce admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.232743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 778/1558] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:14:41.233370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81466a54-554c-4bf4-a5e7-cc5356229f4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f04d65b2185b4f10a065ae5bf24245ec, user_id=13ae9af323874e0eb4807f02c9d6ed31, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.234049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342fb02-73ed-4444-90a2-32fd19cad5ce admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.236072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81466a54-554c-4bf4-a5e7-cc5356229f4a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.244624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342fb02-73ed-4444-90a2-32fd19cad5ce admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.249776 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6056723-ea89-455e-b6b3-67e4c927e47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.250545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056723-ea89-455e-b6b3-67e4c927e47f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.250834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056723-ea89-455e-b6b3-67e4c927e47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.251111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056723-ea89-455e-b6b3-67e4c927e47f admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.251811 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6056723-ea89-455e-b6b3-67e4c927e47f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.252355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 781/1559] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:41 2022] PUT /identity/v3/projects/f04d65b2185b4f10a065ae5bf24245ec/users/13ae9af323874e0eb4807f02c9d6ed31/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:41.252818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-e6056723-ea89-455e-b6b3-67e4c927e47f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:14:41.252818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 04 09:14:41.252818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: Jul 04 09:14:41.253149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81466a54-554c-4bf4-a5e7-cc5356229f4a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.254100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6056723-ea89-455e-b6b3-67e4c927e47f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.285528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 779/1560] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:41.285528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e8dcd-22ca-4152-9cf0-bb6f990904a9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.285528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e8dcd-22ca-4152-9cf0-bb6f990904a9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.285528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e8dcd-22ca-4152-9cf0-bb6f990904a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.301403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-af6e8dcd-22ca-4152-9cf0-bb6f990904a9 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.302661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-af6e8dcd-22ca-4152-9cf0-bb6f990904a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.322305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81466a54-554c-4bf4-a5e7-cc5356229f4a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.348123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:41.365119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 780/1561] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.366696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6432c-7689-407b-94f7-57669cadad83 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.367159 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6432c-7689-407b-94f7-57669cadad83 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.367445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6432c-7689-407b-94f7-57669cadad83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.373446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:41.381549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be681453-b62a-42da-aa7e-761cbee70d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.382504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be681453-b62a-42da-aa7e-761cbee70d11 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.383198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be681453-b62a-42da-aa7e-761cbee70d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.383695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be681453-b62a-42da-aa7e-761cbee70d11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.384788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be681453-b62a-42da-aa7e-761cbee70d11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:14:41.385112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be681453-b62a-42da-aa7e-761cbee70d11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.386538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2fd6432c-7689-407b-94f7-57669cadad83 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.386972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be681453-b62a-42da-aa7e-761cbee70d11 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.387768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2fd6432c-7689-407b-94f7-57669cadad83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.395891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 781/1562] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.396676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6432c-7689-407b-94f7-57669cadad83 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.406530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 782/1563] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:14:41.406530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bc7e3-b568-424e-b4df-9a234f8487d5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.406530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bc7e3-b568-424e-b4df-9a234f8487d5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.406530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bc7e3-b568-424e-b4df-9a234f8487d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.406968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6432c-7689-407b-94f7-57669cadad83 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.411001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c8bdf-b777-4a8c-94db-2cc9d30565a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.411668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c8bdf-b777-4a8c-94db-2cc9d30565a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.412057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c8bdf-b777-4a8c-94db-2cc9d30565a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.412336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c8bdf-b777-4a8c-94db-2cc9d30565a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.413065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c8bdf-b777-4a8c-94db-2cc9d30565a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:14:41.413371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c8bdf-b777-4a8c-94db-2cc9d30565a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.415009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c8bdf-b777-4a8c-94db-2cc9d30565a6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.419629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d34bc7e3-b568-424e-b4df-9a234f8487d5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.420828 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d34bc7e3-b568-424e-b4df-9a234f8487d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.434157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 782/1564] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:14:41.434157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c8bdf-b777-4a8c-94db-2cc9d30565a6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.434453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 783/1565] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.434453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479612e7-1116-400c-9e62-e544fb9eefd9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.434453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479612e7-1116-400c-9e62-e544fb9eefd9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.434453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479612e7-1116-400c-9e62-e544fb9eefd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.439935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c8bdf-b777-4a8c-94db-2cc9d30565a6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.444557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6749f-7735-4ca2-b9fe-69fe3b9bdf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.445220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6749f-7735-4ca2-b9fe-69fe3b9bdf85 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.445507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6749f-7735-4ca2-b9fe-69fe3b9bdf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.445783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6749f-7735-4ca2-b9fe-69fe3b9bdf85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.446543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6749f-7735-4ca2-b9fe-69fe3b9bdf85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.448631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6749f-7735-4ca2-b9fe-69fe3b9bdf85 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.460290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-479612e7-1116-400c-9e62-e544fb9eefd9 None None] MFA Rules not processed for user `13ae9af323874e0eb4807f02c9d6ed31`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.461526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-479612e7-1116-400c-9e62-e544fb9eefd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.475393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 783/1566] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:41.476264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6749f-7735-4ca2-b9fe-69fe3b9bdf85 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.528023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 784/1567] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.529415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224da4-0323-493d-b873-21beb3e664ba None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.529745 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224da4-0323-493d-b873-21beb3e664ba None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.530064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224da4-0323-493d-b873-21beb3e664ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.539987 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6749f-7735-4ca2-b9fe-69fe3b9bdf85 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.544950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182dd606-50c5-4698-9079-758c2c5fbf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.545629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182dd606-50c5-4698-9079-758c2c5fbf10 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.545929 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182dd606-50c5-4698-9079-758c2c5fbf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.546214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182dd606-50c5-4698-9079-758c2c5fbf10 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.546919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182dd606-50c5-4698-9079-758c2c5fbf10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.548559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-05224da4-0323-493d-b873-21beb3e664ba None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:41.549061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182dd606-50c5-4698-9079-758c2c5fbf10 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.549766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-05224da4-0323-493d-b873-21beb3e664ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:41.558136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 785/1568] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:41.558931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05224da4-0323-493d-b873-21beb3e664ba None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.571408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05224da4-0323-493d-b873-21beb3e664ba None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.573403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 784/1569] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:41.574310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182dd606-50c5-4698-9079-758c2c5fbf10 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.576168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189a943-1412-45b8-bb56-a3197dd8c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.576827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189a943-1412-45b8-bb56-a3197dd8c30e admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.577110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189a943-1412-45b8-bb56-a3197dd8c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.577421 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189a943-1412-45b8-bb56-a3197dd8c30e admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.578187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189a943-1412-45b8-bb56-a3197dd8c30e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:14:41.578494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189a943-1412-45b8-bb56-a3197dd8c30e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.580163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189a943-1412-45b8-bb56-a3197dd8c30e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.583909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182dd606-50c5-4698-9079-758c2c5fbf10 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.589302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55be61-0c82-4877-bf74-18b11a24cf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.590063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55be61-0c82-4877-bf74-18b11a24cf3a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.590366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55be61-0c82-4877-bf74-18b11a24cf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.590673 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55be61-0c82-4877-bf74-18b11a24cf3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.591389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55be61-0c82-4877-bf74-18b11a24cf3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.599308 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 786/1570] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:14:41.599308 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189a943-1412-45b8-bb56-a3197dd8c30e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.599579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55be61-0c82-4877-bf74-18b11a24cf3a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.602913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189a943-1412-45b8-bb56-a3197dd8c30e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.607733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab58c9-7a70-4b96-b8b6-00234c40dc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.608348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab58c9-7a70-4b96-b8b6-00234c40dc4c admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.608789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab58c9-7a70-4b96-b8b6-00234c40dc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.609103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab58c9-7a70-4b96-b8b6-00234c40dc4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.609827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab58c9-7a70-4b96-b8b6-00234c40dc4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:14:41.610143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab58c9-7a70-4b96-b8b6-00234c40dc4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.611760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab58c9-7a70-4b96-b8b6-00234c40dc4c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.632693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 787/1571] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:14:41.633254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 785/1572] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:41.633469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab58c9-7a70-4b96-b8b6-00234c40dc4c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.633648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55be61-0c82-4877-bf74-18b11a24cf3a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.643130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab58c9-7a70-4b96-b8b6-00234c40dc4c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.697168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55be61-0c82-4877-bf74-18b11a24cf3a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.698866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4e2e2-5440-4c91-affd-9d92ac69f757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.699622 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4e2e2-5440-4c91-affd-9d92ac69f757 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.699908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4e2e2-5440-4c91-affd-9d92ac69f757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.700183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4e2e2-5440-4c91-affd-9d92ac69f757 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.700882 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4e2e2-5440-4c91-affd-9d92ac69f757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.702618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43238f-b0c5-4bf7-b5e7-724d0cd622ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.702848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4e2e2-5440-4c91-affd-9d92ac69f757 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.703292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43238f-b0c5-4bf7-b5e7-724d0cd622ff admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.703579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43238f-b0c5-4bf7-b5e7-724d0cd622ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.703859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43238f-b0c5-4bf7-b5e7-724d0cd622ff admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.704558 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43238f-b0c5-4bf7-b5e7-724d0cd622ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.706547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43238f-b0c5-4bf7-b5e7-724d0cd622ff admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.728590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 786/1573] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:41.729322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43238f-b0c5-4bf7-b5e7-724d0cd622ff admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.731174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 788/1574] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:41.732438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4e2e2-5440-4c91-affd-9d92ac69f757 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.742939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43238f-b0c5-4bf7-b5e7-724d0cd622ff admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.747496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:41.763635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:41.799244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4e2e2-5440-4c91-affd-9d92ac69f757 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.802606 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012873c2-ed7a-4c82-a861-bf15a42b4517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.803278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012873c2-ed7a-4c82-a861-bf15a42b4517 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.803568 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012873c2-ed7a-4c82-a861-bf15a42b4517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.803841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012873c2-ed7a-4c82-a861-bf15a42b4517 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.804019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569dd644-4bd2-4b67-9f9a-a86834c305b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.804535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012873c2-ed7a-4c82-a861-bf15a42b4517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.804687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dd644-4bd2-4b67-9f9a-a86834c305b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.804974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dd644-4bd2-4b67-9f9a-a86834c305b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.805254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dd644-4bd2-4b67-9f9a-a86834c305b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.806053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569dd644-4bd2-4b67-9f9a-a86834c305b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:14:41.806368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569dd644-4bd2-4b67-9f9a-a86834c305b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.806497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012873c2-ed7a-4c82-a861-bf15a42b4517 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.808580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569dd644-4bd2-4b67-9f9a-a86834c305b0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.819913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 789/1575] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:14:41.820991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569dd644-4bd2-4b67-9f9a-a86834c305b0 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.831195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569dd644-4bd2-4b67-9f9a-a86834c305b0 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.836521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04df5d5-6a07-427c-8731-f7a01cc314d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.837548 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04df5d5-6a07-427c-8731-f7a01cc314d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.838025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04df5d5-6a07-427c-8731-f7a01cc314d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.838305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04df5d5-6a07-427c-8731-f7a01cc314d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.839218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 787/1576] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/projects => generated 357 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:41.839523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04df5d5-6a07-427c-8731-f7a01cc314d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.841068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04df5d5-6a07-427c-8731-f7a01cc314d1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.842925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012873c2-ed7a-4c82-a861-bf15a42b4517 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.857014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 790/1577] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:41.858115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04df5d5-6a07-427c-8731-f7a01cc314d1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.868225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04df5d5-6a07-427c-8731-f7a01cc314d1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.905685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012873c2-ed7a-4c82-a861-bf15a42b4517 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:41.910366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d3b98-0447-4e1f-b345-8d6dc101c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.911031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d3b98-0447-4e1f-b345-8d6dc101c296 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.911318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d3b98-0447-4e1f-b345-8d6dc101c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.911599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d3b98-0447-4e1f-b345-8d6dc101c296 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.912298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d3b98-0447-4e1f-b345-8d6dc101c296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.914261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d3b98-0447-4e1f-b345-8d6dc101c296 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.923403 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d30ed-1379-4a44-88f1-44f5b2dae73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:41.924106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d30ed-1379-4a44-88f1-44f5b2dae73d admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:41.924402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d30ed-1379-4a44-88f1-44f5b2dae73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:41.924672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d30ed-1379-4a44-88f1-44f5b2dae73d admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:41.925360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d30ed-1379-4a44-88f1-44f5b2dae73d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:41.926437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-aa7d30ed-1379-4a44-88f1-44f5b2dae73d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:14:41.926437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jul 04 09:14:41.926437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: Jul 04 09:14:41.927665 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d30ed-1379-4a44-88f1-44f5b2dae73d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:41.942993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 788/1578] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:41.944023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d3b98-0447-4e1f-b345-8d6dc101c296 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.953363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 791/1579] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:41.954472 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d30ed-1379-4a44-88f1-44f5b2dae73d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:41.964265 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d30ed-1379-4a44-88f1-44f5b2dae73d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.008801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d3b98-0447-4e1f-b345-8d6dc101c296 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.015976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35018b-5f71-4de2-b626-fd6a14bb4ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.016901 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35018b-5f71-4de2-b626-fd6a14bb4ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.017189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35018b-5f71-4de2-b626-fd6a14bb4ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.017466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35018b-5f71-4de2-b626-fd6a14bb4ea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.018209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35018b-5f71-4de2-b626-fd6a14bb4ea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.020142 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35018b-5f71-4de2-b626-fd6a14bb4ea0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.021748 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8e01d-6aca-4340-b693-c01c97e85b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.022464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8e01d-6aca-4340-b693-c01c97e85b46 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.022751 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8e01d-6aca-4340-b693-c01c97e85b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.023031 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8e01d-6aca-4340-b693-c01c97e85b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.023766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8e01d-6aca-4340-b693-c01c97e85b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.025364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8e01d-6aca-4340-b693-c01c97e85b46 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.038067 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 792/1580] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.039177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8e01d-6aca-4340-b693-c01c97e85b46 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.050299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8e01d-6aca-4340-b693-c01c97e85b46 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.053355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 789/1581] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:41 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:42.054203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35018b-5f71-4de2-b626-fd6a14bb4ea0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.055764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f2fc6-0c28-4552-887f-c038b457aa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.056659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f2fc6-0c28-4552-887f-c038b457aa5c admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.057144 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f2fc6-0c28-4552-887f-c038b457aa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.057628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f2fc6-0c28-4552-887f-c038b457aa5c admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.058337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:42.058529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f2fc6-0c28-4552-887f-c038b457aa5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.060393 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f2fc6-0c28-4552-887f-c038b457aa5c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.084599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 793/1582] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:42 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:42.084599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f2fc6-0c28-4552-887f-c038b457aa5c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.092245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f2fc6-0c28-4552-887f-c038b457aa5c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.104452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:42.118027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35018b-5f71-4de2-b626-fd6a14bb4ea0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.123044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7689e8b-efef-4d30-b5cf-fb2f32ba3577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.123713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7689e8b-efef-4d30-b5cf-fb2f32ba3577 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.123999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7689e8b-efef-4d30-b5cf-fb2f32ba3577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.124283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7689e8b-efef-4d30-b5cf-fb2f32ba3577 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.124985 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7689e8b-efef-4d30-b5cf-fb2f32ba3577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.126966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7689e8b-efef-4d30-b5cf-fb2f32ba3577 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.151476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd1487-0b11-47a7-82ae-03755c9c004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.152422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd1487-0b11-47a7-82ae-03755c9c004f admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.152739 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd1487-0b11-47a7-82ae-03755c9c004f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.152991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd1487-0b11-47a7-82ae-03755c9c004f admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.153731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd1487-0b11-47a7-82ae-03755c9c004f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.155706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd1487-0b11-47a7-82ae-03755c9c004f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.163488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 790/1583] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:42 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:42.163488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7689e8b-efef-4d30-b5cf-fb2f32ba3577 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.185079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 794/1584] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:42 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:42.185079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd1487-0b11-47a7-82ae-03755c9c004f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.192642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd1487-0b11-47a7-82ae-03755c9c004f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.225991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7689e8b-efef-4d30-b5cf-fb2f32ba3577 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.230837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca040aa-8f64-4d5c-96d7-bbb14ea2448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.231519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca040aa-8f64-4d5c-96d7-bbb14ea2448b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.231787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca040aa-8f64-4d5c-96d7-bbb14ea2448b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.232060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca040aa-8f64-4d5c-96d7-bbb14ea2448b admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.232802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca040aa-8f64-4d5c-96d7-bbb14ea2448b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.234432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca040aa-8f64-4d5c-96d7-bbb14ea2448b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.247791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 791/1585] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.247791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca040aa-8f64-4d5c-96d7-bbb14ea2448b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.254528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130243ce-3355-4aad-bc22-4920a2ae4ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.255030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca040aa-8f64-4d5c-96d7-bbb14ea2448b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.255339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130243ce-3355-4aad-bc22-4920a2ae4ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.255624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130243ce-3355-4aad-bc22-4920a2ae4ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.255900 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130243ce-3355-4aad-bc22-4920a2ae4ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.257009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130243ce-3355-4aad-bc22-4920a2ae4ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.259362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130243ce-3355-4aad-bc22-4920a2ae4ca8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.260394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb567328-de3f-4d1a-8979-2b00890cedaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.261156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb567328-de3f-4d1a-8979-2b00890cedaf admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.261436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb567328-de3f-4d1a-8979-2b00890cedaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.261705 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb567328-de3f-4d1a-8979-2b00890cedaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.262444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb567328-de3f-4d1a-8979-2b00890cedaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.264027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb567328-de3f-4d1a-8979-2b00890cedaf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.274513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 792/1586] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.275321 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb567328-de3f-4d1a-8979-2b00890cedaf admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.281176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 795/1587] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:42 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:42.282009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130243ce-3355-4aad-bc22-4920a2ae4ca8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.290112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb567328-de3f-4d1a-8979-2b00890cedaf admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.291400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130243ce-3355-4aad-bc22-4920a2ae4ca8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.294814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29417dd9-a477-497b-b044-8b192d58cb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.295473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29417dd9-a477-497b-b044-8b192d58cb4f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.295756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29417dd9-a477-497b-b044-8b192d58cb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.296029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29417dd9-a477-497b-b044-8b192d58cb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.296175 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab883c06-5011-4f9e-bf09-e718f06a58cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.296743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29417dd9-a477-497b-b044-8b192d58cb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.296853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab883c06-5011-4f9e-bf09-e718f06a58cf admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.297083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab883c06-5011-4f9e-bf09-e718f06a58cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.297358 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab883c06-5011-4f9e-bf09-e718f06a58cf admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.298070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab883c06-5011-4f9e-bf09-e718f06a58cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.298412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29417dd9-a477-497b-b044-8b192d58cb4f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.299979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab883c06-5011-4f9e-bf09-e718f06a58cf admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.312298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 793/1588] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.312298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29417dd9-a477-497b-b044-8b192d58cb4f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.322311 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29417dd9-a477-497b-b044-8b192d58cb4f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.322760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 796/1589] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:42 2022] POST /identity/v3/users => generated 289 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:42.323507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab883c06-5011-4f9e-bf09-e718f06a58cf admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.330370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cff8ce-c072-4e36-8a2f-43cdd1e06b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.331036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff8ce-c072-4e36-8a2f-43cdd1e06b65 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.331315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff8ce-c072-4e36-8a2f-43cdd1e06b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.331589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cff8ce-c072-4e36-8a2f-43cdd1e06b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.332314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cff8ce-c072-4e36-8a2f-43cdd1e06b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.333508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab883c06-5011-4f9e-bf09-e718f06a58cf admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.333972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cff8ce-c072-4e36-8a2f-43cdd1e06b65 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.338182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbd873-70cc-435b-add3-b97b8fc31164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.338822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbd873-70cc-435b-add3-b97b8fc31164 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.339132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbd873-70cc-435b-add3-b97b8fc31164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.339406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbd873-70cc-435b-add3-b97b8fc31164 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.340094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbd873-70cc-435b-add3-b97b8fc31164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.342064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbd873-70cc-435b-add3-b97b8fc31164 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.347579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 794/1590] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.347579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cff8ce-c072-4e36-8a2f-43cdd1e06b65 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.366826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cff8ce-c072-4e36-8a2f-43cdd1e06b65 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.373554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005d11b-05ab-40ab-8836-2b32eefc8f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.374428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005d11b-05ab-40ab-8836-2b32eefc8f19 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.374715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005d11b-05ab-40ab-8836-2b32eefc8f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.374964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005d11b-05ab-40ab-8836-2b32eefc8f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.375730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005d11b-05ab-40ab-8836-2b32eefc8f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.380528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005d11b-05ab-40ab-8836-2b32eefc8f19 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.380689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 797/1591] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:42 2022] POST /identity/v3/users => generated 287 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:42.380689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbd873-70cc-435b-add3-b97b8fc31164 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.388449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 795/1592] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.392191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbd873-70cc-435b-add3-b97b8fc31164 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.392364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005d11b-05ab-40ab-8836-2b32eefc8f19 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.397580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8db0fd-ce3e-4ef3-8c7d-bc4317b5cfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.398510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8db0fd-ce3e-4ef3-8c7d-bc4317b5cfcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.398792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8db0fd-ce3e-4ef3-8c7d-bc4317b5cfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.399071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8db0fd-ce3e-4ef3-8c7d-bc4317b5cfcf admin admin] PATH_INFO: `/v3/projects/95d5c8e7c27d4e02972278937d08900c/users/5dd5eb59b183436099fb3d605ca421b1/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.402937 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005d11b-05ab-40ab-8836-2b32eefc8f19 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.408041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342968ae-0e51-4b2b-9fdd-d5e91f5b4cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.408692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342968ae-0e51-4b2b-9fdd-d5e91f5b4cec admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.408979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342968ae-0e51-4b2b-9fdd-d5e91f5b4cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.409263 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342968ae-0e51-4b2b-9fdd-d5e91f5b4cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.410045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342968ae-0e51-4b2b-9fdd-d5e91f5b4cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.411673 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342968ae-0e51-4b2b-9fdd-d5e91f5b4cec admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.415189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8db0fd-ce3e-4ef3-8c7d-bc4317b5cfcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d5c8e7c27d4e02972278937d08900c, user_id=5dd5eb59b183436099fb3d605ca421b1, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.418090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8db0fd-ce3e-4ef3-8c7d-bc4317b5cfcf admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.423108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 796/1593] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.423960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342968ae-0e51-4b2b-9fdd-d5e91f5b4cec admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.434583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342968ae-0e51-4b2b-9fdd-d5e91f5b4cec admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.435889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 798/1594] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:42 2022] PUT /identity/v3/projects/95d5c8e7c27d4e02972278937d08900c/users/5dd5eb59b183436099fb3d605ca421b1/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:42.436893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8db0fd-ce3e-4ef3-8c7d-bc4317b5cfcf admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.472810 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:42.481438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:42.503552 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8db0fd-ce3e-4ef3-8c7d-bc4317b5cfcf admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.507330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3bbe4-f9a1-4c3a-8d9c-1a45b23d977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.508455 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3bbe4-f9a1-4c3a-8d9c-1a45b23d977a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.508804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdce940-d953-4828-9805-f21f1441feb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.508937 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3bbe4-f9a1-4c3a-8d9c-1a45b23d977a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.509230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3bbe4-f9a1-4c3a-8d9c-1a45b23d977a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.509813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdce940-d953-4828-9805-f21f1441feb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.510081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdce940-d953-4828-9805-f21f1441feb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.510210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3bbe4-f9a1-4c3a-8d9c-1a45b23d977a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.510359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdce940-d953-4828-9805-f21f1441feb0 admin admin] PATH_INFO: `/v3/projects/13c95b6f91e84cd498cc57c55253a477/users/cb99f1e73a214b4ab32102a9ab5670fb/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.512113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3bbe4-f9a1-4c3a-8d9c-1a45b23d977a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.525007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 797/1595] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.525007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3bbe4-f9a1-4c3a-8d9c-1a45b23d977a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.525255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdce940-d953-4828-9805-f21f1441feb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c95b6f91e84cd498cc57c55253a477, user_id=cb99f1e73a214b4ab32102a9ab5670fb, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.527722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdce940-d953-4828-9805-f21f1441feb0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.533565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3bbe4-f9a1-4c3a-8d9c-1a45b23d977a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.538928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415ed6c-581e-43b2-be01-1306161b4616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.539614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415ed6c-581e-43b2-be01-1306161b4616 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.539887 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415ed6c-581e-43b2-be01-1306161b4616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.540166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415ed6c-581e-43b2-be01-1306161b4616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.540897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415ed6c-581e-43b2-be01-1306161b4616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.542079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 799/1596] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:42 2022] PUT /identity/v3/projects/13c95b6f91e84cd498cc57c55253a477/users/cb99f1e73a214b4ab32102a9ab5670fb/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:42.542798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415ed6c-581e-43b2-be01-1306161b4616 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.543061 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdce940-d953-4828-9805-f21f1441feb0 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.554244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 798/1597] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.555377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415ed6c-581e-43b2-be01-1306161b4616 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.567140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415ed6c-581e-43b2-be01-1306161b4616 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.605462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdce940-d953-4828-9805-f21f1441feb0 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.610087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92997ba7-f190-4431-93dd-5197e6678082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.610762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92997ba7-f190-4431-93dd-5197e6678082 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.611046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92997ba7-f190-4431-93dd-5197e6678082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.611328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92997ba7-f190-4431-93dd-5197e6678082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.612058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92997ba7-f190-4431-93dd-5197e6678082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.613682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92997ba7-f190-4431-93dd-5197e6678082 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.623614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 800/1598] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.623891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac4a1b-352c-4707-90d7-3b61cd490580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.624399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92997ba7-f190-4431-93dd-5197e6678082 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.624579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac4a1b-352c-4707-90d7-3b61cd490580 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.624855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac4a1b-352c-4707-90d7-3b61cd490580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.625136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac4a1b-352c-4707-90d7-3b61cd490580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.625868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac4a1b-352c-4707-90d7-3b61cd490580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.627536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac4a1b-352c-4707-90d7-3b61cd490580 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.636474 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92997ba7-f190-4431-93dd-5197e6678082 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.641189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 799/1599] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.641189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac4a1b-352c-4707-90d7-3b61cd490580 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.641397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094823b-83b4-4aa9-8e3e-6c51dee3ebc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.641985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094823b-83b4-4aa9-8e3e-6c51dee3ebc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.642275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094823b-83b4-4aa9-8e3e-6c51dee3ebc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.642569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094823b-83b4-4aa9-8e3e-6c51dee3ebc8 admin admin] PATH_INFO: `/v3/projects/eb49ab75fe7d434c95d639cbd0bcb634/users/1268529f491a466882effe0456cccb1b/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.648454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac4a1b-352c-4707-90d7-3b61cd490580 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.653669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5947b9-b4d0-475b-b1fb-5b3a102d0e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.654582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5947b9-b4d0-475b-b1fb-5b3a102d0e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.655021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5947b9-b4d0-475b-b1fb-5b3a102d0e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.655310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5947b9-b4d0-475b-b1fb-5b3a102d0e9d admin admin] PATH_INFO: `/v3/projects/48690ebc49824572a08f53597eeee661/users/1e916bf18efe421895c8d7b28fcd55d5/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.658560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094823b-83b4-4aa9-8e3e-6c51dee3ebc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb49ab75fe7d434c95d639cbd0bcb634, user_id=1268529f491a466882effe0456cccb1b, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.661292 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094823b-83b4-4aa9-8e3e-6c51dee3ebc8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.674171 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5947b9-b4d0-475b-b1fb-5b3a102d0e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48690ebc49824572a08f53597eeee661, user_id=1e916bf18efe421895c8d7b28fcd55d5, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.675288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-df5947b9-b4d0-475b-b1fb-5b3a102d0e9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:14:42.675288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 04 09:14:42.675288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: Jul 04 09:14:42.677245 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5947b9-b4d0-475b-b1fb-5b3a102d0e9d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.677778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 801/1600] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:42 2022] PUT /identity/v3/projects/eb49ab75fe7d434c95d639cbd0bcb634/users/1268529f491a466882effe0456cccb1b/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:42.679232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa18b8-7785-4013-a9fc-8d97703154dc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.679507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa18b8-7785-4013-a9fc-8d97703154dc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.679799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa18b8-7785-4013-a9fc-8d97703154dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.692181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 800/1601] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:42 2022] PUT /identity/v3/projects/48690ebc49824572a08f53597eeee661/users/1e916bf18efe421895c8d7b28fcd55d5/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:42.693302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5947b9-b4d0-475b-b1fb-5b3a102d0e9d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.710148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-52fa18b8-7785-4013-a9fc-8d97703154dc None None] MFA Rules not processed for user `5dd5eb59b183436099fb3d605ca421b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:42.711439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-52fa18b8-7785-4013-a9fc-8d97703154dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:42.756025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5947b9-b4d0-475b-b1fb-5b3a102d0e9d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.761209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314f8d8-4785-46ae-9b9b-4488c72fb2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.761863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314f8d8-4785-46ae-9b9b-4488c72fb2ae admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.762290 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314f8d8-4785-46ae-9b9b-4488c72fb2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.762571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314f8d8-4785-46ae-9b9b-4488c72fb2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.763472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314f8d8-4785-46ae-9b9b-4488c72fb2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.765124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314f8d8-4785-46ae-9b9b-4488c72fb2ae admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.775474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 801/1602] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.776778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad3109-ba15-4336-b41c-533f7809bb6c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.777067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad3109-ba15-4336-b41c-533f7809bb6c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.777347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad3109-ba15-4336-b41c-533f7809bb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.779966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 802/1603] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:42 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:42.781079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa18b8-7785-4013-a9fc-8d97703154dc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.793579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa18b8-7785-4013-a9fc-8d97703154dc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.795152 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-55ad3109-ba15-4336-b41c-533f7809bb6c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:42.796435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-55ad3109-ba15-4336-b41c-533f7809bb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:42.798299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbeec1-14ce-4317-ba2a-c87c248a76ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.798984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbeec1-14ce-4317-ba2a-c87c248a76ad admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.799267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbeec1-14ce-4317-ba2a-c87c248a76ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.799548 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbeec1-14ce-4317-ba2a-c87c248a76ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.800288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbeec1-14ce-4317-ba2a-c87c248a76ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.801947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbeec1-14ce-4317-ba2a-c87c248a76ad admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.804548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 802/1604] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:42.805302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad3109-ba15-4336-b41c-533f7809bb6c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.815154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 803/1605] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.815872 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbeec1-14ce-4317-ba2a-c87c248a76ad admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.817664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad3109-ba15-4336-b41c-533f7809bb6c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.822479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df80bc-0ec1-4cf7-b43b-dce7fb86643d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.823436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df80bc-0ec1-4cf7-b43b-dce7fb86643d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.823911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df80bc-0ec1-4cf7-b43b-dce7fb86643d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.824260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df80bc-0ec1-4cf7-b43b-dce7fb86643d admin admin] PATH_INFO: `/v3/projects/80a6421aa9a94f0badd40600792880fd/users/ddea91415fb54f29a18db59ca41085aa/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.825478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbeec1-14ce-4317-ba2a-c87c248a76ad admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.830087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6785c-b2cc-43f2-8ac8-3e32b1dcd516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.830732 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6785c-b2cc-43f2-8ac8-3e32b1dcd516 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.831015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6785c-b2cc-43f2-8ac8-3e32b1dcd516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.831289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6785c-b2cc-43f2-8ac8-3e32b1dcd516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.832129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6785c-b2cc-43f2-8ac8-3e32b1dcd516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.833699 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6785c-b2cc-43f2-8ac8-3e32b1dcd516 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.837343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:42.840040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:42.840633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df80bc-0ec1-4cf7-b43b-dce7fb86643d admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a6421aa9a94f0badd40600792880fd, user_id=ddea91415fb54f29a18db59ca41085aa, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.843477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df80bc-0ec1-4cf7-b43b-dce7fb86643d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.846983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 804/1606] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.846983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6785c-b2cc-43f2-8ac8-3e32b1dcd516 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.857525 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6785c-b2cc-43f2-8ac8-3e32b1dcd516 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.861245 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 803/1607] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:42 2022] PUT /identity/v3/projects/80a6421aa9a94f0badd40600792880fd/users/ddea91415fb54f29a18db59ca41085aa/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:42.862635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7a438-7577-44d0-b948-0a059e1e93c8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.862927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7a438-7577-44d0-b948-0a059e1e93c8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.863211 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7a438-7577-44d0-b948-0a059e1e93c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.880677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3ea7a438-7577-44d0-b948-0a059e1e93c8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:42.881910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3ea7a438-7577-44d0-b948-0a059e1e93c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:42.924269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c691b-cf52-4913-8a19-664e36db405f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.925089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c691b-cf52-4913-8a19-664e36db405f admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.925238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c691b-cf52-4913-8a19-664e36db405f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.925509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c691b-cf52-4913-8a19-664e36db405f admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.926324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c691b-cf52-4913-8a19-664e36db405f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.927963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c691b-cf52-4913-8a19-664e36db405f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.938528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 805/1608] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:42.939099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 804/1609] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:42.939840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7a438-7577-44d0-b948-0a059e1e93c8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.942838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c691b-cf52-4913-8a19-664e36db405f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.950548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7a438-7577-44d0-b948-0a059e1e93c8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.954023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c691b-cf52-4913-8a19-664e36db405f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:42.955372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671fab2-bb56-4239-98f7-4b5cb11975bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.956105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671fab2-bb56-4239-98f7-4b5cb11975bb admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.956543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671fab2-bb56-4239-98f7-4b5cb11975bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.956891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671fab2-bb56-4239-98f7-4b5cb11975bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.957657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671fab2-bb56-4239-98f7-4b5cb11975bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.958722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94af95-14c4-4935-a6d1-6ced857d54f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:42.959632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94af95-14c4-4935-a6d1-6ced857d54f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:42.959741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671fab2-bb56-4239-98f7-4b5cb11975bb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.959919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94af95-14c4-4935-a6d1-6ced857d54f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:42.960201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94af95-14c4-4935-a6d1-6ced857d54f4 admin admin] PATH_INFO: `/v3/projects/13eb5bc141c34f99ab7b090ff2ea1146/users/cc12f51acc3945d08bc1fb277f77ec67/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:42.975177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94af95-14c4-4935-a6d1-6ced857d54f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13eb5bc141c34f99ab7b090ff2ea1146, user_id=cc12f51acc3945d08bc1fb277f77ec67, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:42.977961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94af95-14c4-4935-a6d1-6ced857d54f4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:42.987373 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 805/1610] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:42 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:42.988248 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671fab2-bb56-4239-98f7-4b5cb11975bb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:42.991415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 806/1611] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:42 2022] PUT /identity/v3/projects/13eb5bc141c34f99ab7b090ff2ea1146/users/cc12f51acc3945d08bc1fb277f77ec67/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:42.992191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94af95-14c4-4935-a6d1-6ced857d54f4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:43.049663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671fab2-bb56-4239-98f7-4b5cb11975bb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:43.053450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70692ce4-90ca-4e58-ac38-19a49ea6ac53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:43.054140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70692ce4-90ca-4e58-ac38-19a49ea6ac53 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.054426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70692ce4-90ca-4e58-ac38-19a49ea6ac53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.054702 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70692ce4-90ca-4e58-ac38-19a49ea6ac53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.065948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94af95-14c4-4935-a6d1-6ced857d54f4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:43.116846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70692ce4-90ca-4e58-ac38-19a49ea6ac53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:43.121778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6773c8-5b35-4dfc-86dc-b30d898d403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:43.122674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70692ce4-90ca-4e58-ac38-19a49ea6ac53 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:43.122797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6773c8-5b35-4dfc-86dc-b30d898d403c admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.123202 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6773c8-5b35-4dfc-86dc-b30d898d403c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.123704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6773c8-5b35-4dfc-86dc-b30d898d403c admin admin] PATH_INFO: `/v3/projects/8cfd9d65f66641b2bda6bbdcf908ca3c/users/43bcd8fb25604011bfd1df823ea4ddad/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.140087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 806/1612] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:43.141282 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70692ce4-90ca-4e58-ac38-19a49ea6ac53 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:43.146656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6773c8-5b35-4dfc-86dc-b30d898d403c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cfd9d65f66641b2bda6bbdcf908ca3c, user_id=43bcd8fb25604011bfd1df823ea4ddad, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:43.150215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6773c8-5b35-4dfc-86dc-b30d898d403c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:43.153210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70692ce4-90ca-4e58-ac38-19a49ea6ac53 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:43.161235 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f092f6-6f58-45f8-b79f-b4e5d2b522ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:43.162564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f092f6-6f58-45f8-b79f-b4e5d2b522ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.163025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f092f6-6f58-45f8-b79f-b4e5d2b522ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.163618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f092f6-6f58-45f8-b79f-b4e5d2b522ca admin admin] PATH_INFO: `/v3/projects/b811e2ed889f4b9c9a776e88971ef58a/users/707c9346e07d4fa29aabfab72b0f93db/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.178633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 807/1613] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:42 2022] PUT /identity/v3/projects/8cfd9d65f66641b2bda6bbdcf908ca3c/users/43bcd8fb25604011bfd1df823ea4ddad/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:43.178633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92349f5-6ece-43ec-a2a4-131aa1792025 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.178633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92349f5-6ece-43ec-a2a4-131aa1792025 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.178633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92349f5-6ece-43ec-a2a4-131aa1792025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.188102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:43.195852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f092f6-6f58-45f8-b79f-b4e5d2b522ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=b811e2ed889f4b9c9a776e88971ef58a, user_id=707c9346e07d4fa29aabfab72b0f93db, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:43.201687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f092f6-6f58-45f8-b79f-b4e5d2b522ca admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:43.207251 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:43.220191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b92349f5-6ece-43ec-a2a4-131aa1792025 None None] MFA Rules not processed for user `ddea91415fb54f29a18db59ca41085aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:43.222223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b92349f5-6ece-43ec-a2a4-131aa1792025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:43.237922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 807/1614] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:43 2022] PUT /identity/v3/projects/b811e2ed889f4b9c9a776e88971ef58a/users/707c9346e07d4fa29aabfab72b0f93db/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:43.237922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f092f6-6f58-45f8-b79f-b4e5d2b522ca admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:43.310188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f092f6-6f58-45f8-b79f-b4e5d2b522ca admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:43.313117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 808/1615] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:43.314392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92349f5-6ece-43ec-a2a4-131aa1792025 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:43.315493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8cd84-710a-4e92-9cfe-ada1019b790e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:43.315972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b92349f5-6ece-43ec-a2a4-131aa1792025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:43.316405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8cd84-710a-4e92-9cfe-ada1019b790e admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.316687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8cd84-710a-4e92-9cfe-ada1019b790e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.316960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8cd84-710a-4e92-9cfe-ada1019b790e admin admin] PATH_INFO: `/v3/projects/eb49ab75fe7d434c95d639cbd0bcb634/users/1268529f491a466882effe0456cccb1b/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.319393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8cd84-710a-4e92-9cfe-ada1019b790e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb49ab75fe7d434c95d639cbd0bcb634, user_id=1268529f491a466882effe0456cccb1b, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:43.322263 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8cd84-710a-4e92-9cfe-ada1019b790e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:43.334888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 808/1616] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:43 2022] PUT /identity/v3/projects/eb49ab75fe7d434c95d639cbd0bcb634/users/1268529f491a466882effe0456cccb1b/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:43.335737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8cd84-710a-4e92-9cfe-ada1019b790e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:43.393876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92349f5-6ece-43ec-a2a4-131aa1792025 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:43.413743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8cd84-710a-4e92-9cfe-ada1019b790e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:43.419367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b22f92-d432-45d5-9b14-af1e5a403779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:43.420190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b22f92-d432-45d5-9b14-af1e5a403779 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.420589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b22f92-d432-45d5-9b14-af1e5a403779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.420872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b22f92-d432-45d5-9b14-af1e5a403779 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.421630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b22f92-d432-45d5-9b14-af1e5a403779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:43.423619 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b22f92-d432-45d5-9b14-af1e5a403779 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:43.451215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 809/1617] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:43 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:43.451215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5670401-aac2-4a26-92e6-2a823b024489 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.451215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5670401-aac2-4a26-92e6-2a823b024489 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.451215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5670401-aac2-4a26-92e6-2a823b024489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.463219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a312d87-4d92-47b6-9370-13850754f84e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:43.463911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a312d87-4d92-47b6-9370-13850754f84e service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.464262 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a312d87-4d92-47b6-9370-13850754f84e service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.464588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a312d87-4d92-47b6-9370-13850754f84e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.474921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a312d87-4d92-47b6-9370-13850754f84e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:43.476657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a312d87-4d92-47b6-9370-13850754f84e service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:43.483585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d5670401-aac2-4a26-92e6-2a823b024489 None None] MFA Rules not processed for user `cc12f51acc3945d08bc1fb277f77ec67`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:43.484969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d5670401-aac2-4a26-92e6-2a823b024489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:43.489469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 809/1618] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:43.490834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24296b-72c5-4f5b-9052-13e4b126a93e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.491260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24296b-72c5-4f5b-9052-13e4b126a93e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.491645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24296b-72c5-4f5b-9052-13e4b126a93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.512025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5f24296b-72c5-4f5b-9052-13e4b126a93e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:43.513358 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5f24296b-72c5-4f5b-9052-13e4b126a93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:43.525335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 810/1619] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:43.527148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7aac3-d15f-4c06-97c3-41dd4f63d07e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.527507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7aac3-d15f-4c06-97c3-41dd4f63d07e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.527958 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7aac3-d15f-4c06-97c3-41dd4f63d07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.562927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e7d7aac3-d15f-4c06-97c3-41dd4f63d07e None None] MFA Rules not processed for user `707c9346e07d4fa29aabfab72b0f93db`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:43.564158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e7d7aac3-d15f-4c06-97c3-41dd4f63d07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:43.570777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 810/1620] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:43.572411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055ea15-8dd1-4d01-a64a-bee884ddf406 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.572710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055ea15-8dd1-4d01-a64a-bee884ddf406 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.572990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055ea15-8dd1-4d01-a64a-bee884ddf406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.597548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:43.614675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e055ea15-8dd1-4d01-a64a-bee884ddf406 None None] MFA Rules not processed for user `1268529f491a466882effe0456cccb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:43.616929 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e055ea15-8dd1-4d01-a64a-bee884ddf406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:43.649331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:43.691440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 811/1621] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:43.691440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c468d05-5efd-4f74-900b-c4657797f650 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.691440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c468d05-5efd-4f74-900b-c4657797f650 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.691440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c468d05-5efd-4f74-900b-c4657797f650 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.692764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 812/1622] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 4 09:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:14:43.694754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c468d05-5efd-4f74-900b-c4657797f650 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:43.710237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c468d05-5efd-4f74-900b-c4657797f650 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:43.715838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908670f-f954-4fce-a389-2d9aa2342f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:43.716601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908670f-f954-4fce-a389-2d9aa2342f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.717480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908670f-f954-4fce-a389-2d9aa2342f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.717480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908670f-f954-4fce-a389-2d9aa2342f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.718129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908670f-f954-4fce-a389-2d9aa2342f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:43.720361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908670f-f954-4fce-a389-2d9aa2342f7f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:43.733691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 811/1623] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:43 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:43.733691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055ea15-8dd1-4d01-a64a-bee884ddf406 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:43.734016 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 813/1624] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:43.734178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900df6a1-2adb-4e6c-b3ba-36b5e9832641 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.734563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900df6a1-2adb-4e6c-b3ba-36b5e9832641 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.734939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900df6a1-2adb-4e6c-b3ba-36b5e9832641 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.736578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 814/1625] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 4 09:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:14:43.737512 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900df6a1-2adb-4e6c-b3ba-36b5e9832641 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:43.753456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900df6a1-2adb-4e6c-b3ba-36b5e9832641 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:43.757611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055ea15-8dd1-4d01-a64a-bee884ddf406 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:43.758985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ab6f9-9fb5-4940-93b6-1fb082ad76b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:43.759791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ab6f9-9fb5-4940-93b6-1fb082ad76b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.760151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ab6f9-9fb5-4940-93b6-1fb082ad76b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.760497 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ab6f9-9fb5-4940-93b6-1fb082ad76b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.761404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ab6f9-9fb5-4940-93b6-1fb082ad76b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:43.762545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a56e4-95d7-42a6-9aaf-372600aa04a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:43.763334 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a56e4-95d7-42a6-9aaf-372600aa04a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.763603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ab6f9-9fb5-4940-93b6-1fb082ad76b3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:43.763727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a56e4-95d7-42a6-9aaf-372600aa04a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.764098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a56e4-95d7-42a6-9aaf-372600aa04a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.776855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a56e4-95d7-42a6-9aaf-372600aa04a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:43.778467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a56e4-95d7-42a6-9aaf-372600aa04a9 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:43.790276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 812/1626] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:43.791686 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7290512-421e-460d-bef8-9abff64e4364 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.792081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7290512-421e-460d-bef8-9abff64e4364 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.792511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7290512-421e-460d-bef8-9abff64e4364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.797287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 815/1627] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:43 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:43.797287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ab6f9-9fb5-4940-93b6-1fb082ad76b3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:43.814715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d7290512-421e-460d-bef8-9abff64e4364 None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:43.816310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d7290512-421e-460d-bef8-9abff64e4364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:43.879108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ab6f9-9fb5-4940-93b6-1fb082ad76b3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:43.897132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df8242-eecd-4bbd-8f3a-3e02621ae96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:43.897973 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df8242-eecd-4bbd-8f3a-3e02621ae96e admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.898415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df8242-eecd-4bbd-8f3a-3e02621ae96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.898544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df8242-eecd-4bbd-8f3a-3e02621ae96e admin admin] PATH_INFO: `/v3/projects/470b0b00efba49ecbab24afb7c2c5215/users/70f64e5ae02d44209def12327bcd6e2b/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.902800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 813/1628] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 4 09:14:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:43.904350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8898d6-b928-424f-8702-88509516bbdf None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:43.904765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8898d6-b928-424f-8702-88509516bbdf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:43.905292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8898d6-b928-424f-8702-88509516bbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:43.923926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df8242-eecd-4bbd-8f3a-3e02621ae96e admin admin] RBAC: Authorizing `identity:create_grant(project_id=470b0b00efba49ecbab24afb7c2c5215, user_id=70f64e5ae02d44209def12327bcd6e2b, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:43.926781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df8242-eecd-4bbd-8f3a-3e02621ae96e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:43.928650 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5a8898d6-b928-424f-8702-88509516bbdf None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:43.930116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5a8898d6-b928-424f-8702-88509516bbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:43.940549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 814/1629] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 4 09:14:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:43.948154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 816/1630] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:43 2022] PUT /identity/v3/projects/470b0b00efba49ecbab24afb7c2c5215/users/70f64e5ae02d44209def12327bcd6e2b/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:43.948686 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8898d6-b928-424f-8702-88509516bbdf None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:43.949444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df8242-eecd-4bbd-8f3a-3e02621ae96e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:44.008327 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8898d6-b928-424f-8702-88509516bbdf None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:44.028702 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df8242-eecd-4bbd-8f3a-3e02621ae96e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:44.031052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:44.034457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe00360-9c6a-4b56-bf81-7a5ba1ba4a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:44.035148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe00360-9c6a-4b56-bf81-7a5ba1ba4a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.035860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe00360-9c6a-4b56-bf81-7a5ba1ba4a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.035860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe00360-9c6a-4b56-bf81-7a5ba1ba4a1e admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.036489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe00360-9c6a-4b56-bf81-7a5ba1ba4a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:44.038463 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe00360-9c6a-4b56-bf81-7a5ba1ba4a1e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:44.064623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 817/1631] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:43 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:44.064623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe00360-9c6a-4b56-bf81-7a5ba1ba4a1e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:44.067893 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6175e7-1885-4e97-9ff8-f49348e0a994 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:44.068722 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6175e7-1885-4e97-9ff8-f49348e0a994 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.068860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6175e7-1885-4e97-9ff8-f49348e0a994 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.069114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6175e7-1885-4e97-9ff8-f49348e0a994 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.091380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:44.187911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6175e7-1885-4e97-9ff8-f49348e0a994 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:44.189547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6175e7-1885-4e97-9ff8-f49348e0a994 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:44.199731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe00360-9c6a-4b56-bf81-7a5ba1ba4a1e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:44.203391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 815/1632] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:43 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:44.204665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6175e7-1885-4e97-9ff8-f49348e0a994 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:44.208493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc386a-87ea-40f5-8b6b-1f2cedbb14f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:44.209212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc386a-87ea-40f5-8b6b-1f2cedbb14f6 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.209550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc386a-87ea-40f5-8b6b-1f2cedbb14f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.209830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc386a-87ea-40f5-8b6b-1f2cedbb14f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.223797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6175e7-1885-4e97-9ff8-f49348e0a994 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:44.243849 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2914c28-e6e4-4977-adb8-23a0903b59ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:44.245616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2914c28-e6e4-4977-adb8-23a0903b59ab service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.249747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2914c28-e6e4-4977-adb8-23a0903b59ab service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.250160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2914c28-e6e4-4977-adb8-23a0903b59ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.304631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc386a-87ea-40f5-8b6b-1f2cedbb14f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:44.306458 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc386a-87ea-40f5-8b6b-1f2cedbb14f6 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:44.328390 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 818/1633] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:44.329833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc386a-87ea-40f5-8b6b-1f2cedbb14f6 service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:44.365730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc386a-87ea-40f5-8b6b-1f2cedbb14f6 service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:44.375789 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2914c28-e6e4-4977-adb8-23a0903b59ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:44.378291 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2914c28-e6e4-4977-adb8-23a0903b59ab service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:44.378573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c855754-d6c2-4464-9265-64b9a72370e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:44.379540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855754-d6c2-4464-9265-64b9a72370e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.379860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855754-d6c2-4464-9265-64b9a72370e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.381298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855754-d6c2-4464-9265-64b9a72370e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.384373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c855754-d6c2-4464-9265-64b9a72370e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:44.387810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c855754-d6c2-4464-9265-64b9a72370e9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:44.392082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 816/1634] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:44.403806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2914c28-e6e4-4977-adb8-23a0903b59ab service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:44.416375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 819/1635] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:44.416375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c855754-d6c2-4464-9265-64b9a72370e9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:44.434644 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2914c28-e6e4-4977-adb8-23a0903b59ab service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:44.439305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c855754-d6c2-4464-9265-64b9a72370e9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:44.444522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d273eaa-1d39-4b89-b9e9-1ffa5432458e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:44.446495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d273eaa-1d39-4b89-b9e9-1ffa5432458e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.447111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d273eaa-1d39-4b89-b9e9-1ffa5432458e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.448789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09810995-7bc4-4102-82f8-f713aa447a48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:44.449401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d273eaa-1d39-4b89-b9e9-1ffa5432458e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.449548 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09810995-7bc4-4102-82f8-f713aa447a48 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.449829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09810995-7bc4-4102-82f8-f713aa447a48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.450748 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09810995-7bc4-4102-82f8-f713aa447a48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.450855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d273eaa-1d39-4b89-b9e9-1ffa5432458e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:44.454603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d273eaa-1d39-4b89-b9e9-1ffa5432458e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:44.479069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 817/1636] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:44.482022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d273eaa-1d39-4b89-b9e9-1ffa5432458e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:44.506420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d273eaa-1d39-4b89-b9e9-1ffa5432458e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:44.518957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2bc8b-03c5-45d5-b36d-e68b500a5739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:44.520701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2bc8b-03c5-45d5-b36d-e68b500a5739 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.522050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2bc8b-03c5-45d5-b36d-e68b500a5739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.522440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2bc8b-03c5-45d5-b36d-e68b500a5739 admin admin] PATH_INFO: `/v3/projects/470b0b00efba49ecbab24afb7c2c5215/users/70f64e5ae02d44209def12327bcd6e2b/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.528753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2bc8b-03c5-45d5-b36d-e68b500a5739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=470b0b00efba49ecbab24afb7c2c5215, user_id=70f64e5ae02d44209def12327bcd6e2b, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:44.534814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2bc8b-03c5-45d5-b36d-e68b500a5739 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:44.561135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 818/1637] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:44 2022] PUT /identity/v3/projects/470b0b00efba49ecbab24afb7c2c5215/users/70f64e5ae02d44209def12327bcd6e2b/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:44.562921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2bc8b-03c5-45d5-b36d-e68b500a5739 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:44.578276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09810995-7bc4-4102-82f8-f713aa447a48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:44.579788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09810995-7bc4-4102-82f8-f713aa447a48 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:44.606573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 820/1638] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:44 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:44.607997 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c0508-83d6-46dc-87cb-d88f88e9b2a0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.608285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c0508-83d6-46dc-87cb-d88f88e9b2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.608562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c0508-83d6-46dc-87cb-d88f88e9b2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.673750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e96c0508-83d6-46dc-87cb-d88f88e9b2a0 None None] MFA Rules not processed for user `70f64e5ae02d44209def12327bcd6e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:44.676731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e96c0508-83d6-46dc-87cb-d88f88e9b2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:44.688763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:44.723835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2bc8b-03c5-45d5-b36d-e68b500a5739 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:44.741145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37ed38-9986-4e35-8337-d0783421f482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:44.742678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37ed38-9986-4e35-8337-d0783421f482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.743116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37ed38-9986-4e35-8337-d0783421f482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.743480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37ed38-9986-4e35-8337-d0783421f482 admin admin] PATH_INFO: `/v3/projects/52aedf148cd943a5b8bbe70b19aeea49/users/e122b65fab6e4758b95177c84626abef/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.743894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:44.779232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37ed38-9986-4e35-8337-d0783421f482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52aedf148cd943a5b8bbe70b19aeea49, user_id=e122b65fab6e4758b95177c84626abef, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:44.786328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37ed38-9986-4e35-8337-d0783421f482 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:44.817933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 819/1639] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:44 2022] PUT /identity/v3/projects/52aedf148cd943a5b8bbe70b19aeea49/users/e122b65fab6e4758b95177c84626abef/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:44.819715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a6f1-8a25-4511-9a3c-6bf3f16e19a0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.821170 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a6f1-8a25-4511-9a3c-6bf3f16e19a0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.821586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a6f1-8a25-4511-9a3c-6bf3f16e19a0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.824917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 820/1640] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 4 09:14:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:14:44.825984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892a6f1-8a25-4511-9a3c-6bf3f16e19a0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:44.853781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 821/1641] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:44.855294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec05d2-c30b-403d-9066-2581927bfc57 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.855585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec05d2-c30b-403d-9066-2581927bfc57 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.856035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec05d2-c30b-403d-9066-2581927bfc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.920963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-07ec05d2-c30b-403d-9066-2581927bfc57 None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:44.924288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-07ec05d2-c30b-403d-9066-2581927bfc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:44.986679 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892a6f1-8a25-4511-9a3c-6bf3f16e19a0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:44.991427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dff55-cf7a-4cb1-b020-07043af8f392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:44.992164 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dff55-cf7a-4cb1-b020-07043af8f392 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:44.992452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dff55-cf7a-4cb1-b020-07043af8f392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:44.992729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dff55-cf7a-4cb1-b020-07043af8f392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:44.993614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055dff55-cf7a-4cb1-b020-07043af8f392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:44.996284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055dff55-cf7a-4cb1-b020-07043af8f392 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:45.031429 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 821/1642] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:45.033175 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dff55-cf7a-4cb1-b020-07043af8f392 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:45.150811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 822/1643] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:14:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:45.153683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e643b-6068-449b-b6d7-ff53f00bf1ea None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:45.154035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e643b-6068-449b-b6d7-ff53f00bf1ea None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:45.154319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e643b-6068-449b-b6d7-ff53f00bf1ea None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:45.156293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 823/1644] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 4 09:14:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:14:45.157040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e643b-6068-449b-b6d7-ff53f00bf1ea None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:45.173307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dff55-cf7a-4cb1-b020-07043af8f392 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:45.174584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e643b-6068-449b-b6d7-ff53f00bf1ea None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:45.181800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf00b4-ac34-4b63-8fed-fa0955f27185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:45.187092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da1ae0-0f60-4235-b8c1-35e99ea44ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:45.187255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf00b4-ac34-4b63-8fed-fa0955f27185 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:45.187606 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf00b4-ac34-4b63-8fed-fa0955f27185 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:45.187895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf00b4-ac34-4b63-8fed-fa0955f27185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:45.188024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da1ae0-0f60-4235-b8c1-35e99ea44ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:45.188285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da1ae0-0f60-4235-b8c1-35e99ea44ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:45.188561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da1ae0-0f60-4235-b8c1-35e99ea44ddf admin admin] PATH_INFO: `/v3/projects/52aedf148cd943a5b8bbe70b19aeea49/users/e122b65fab6e4758b95177c84626abef/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:45.195501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da1ae0-0f60-4235-b8c1-35e99ea44ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=52aedf148cd943a5b8bbe70b19aeea49, user_id=e122b65fab6e4758b95177c84626abef, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:45.198531 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da1ae0-0f60-4235-b8c1-35e99ea44ddf admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:45.215553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 824/1645] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:45 2022] PUT /identity/v3/projects/52aedf148cd943a5b8bbe70b19aeea49/users/e122b65fab6e4758b95177c84626abef/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:45.217621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b60dd7-2475-4814-99be-477284517653 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:45.217960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b60dd7-2475-4814-99be-477284517653 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:45.218247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b60dd7-2475-4814-99be-477284517653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:45.243632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-74b60dd7-2475-4814-99be-477284517653 None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:45.245005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-74b60dd7-2475-4814-99be-477284517653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:45.279066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf00b4-ac34-4b63-8fed-fa0955f27185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:45.280573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf00b4-ac34-4b63-8fed-fa0955f27185 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:45.294161 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 822/1646] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:45.295091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf00b4-ac34-4b63-8fed-fa0955f27185 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:45.304614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf00b4-ac34-4b63-8fed-fa0955f27185 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:45.321300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:45.344930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 825/1647] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:45.347074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af4d9c-4c6c-4265-a04b-7a2057193129 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:45.347762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af4d9c-4c6c-4265-a04b-7a2057193129 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:45.348364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af4d9c-4c6c-4265-a04b-7a2057193129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:45.384836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-98af4d9c-4c6c-4265-a04b-7a2057193129 None None] MFA Rules not processed for user `e122b65fab6e4758b95177c84626abef`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:45.386786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-98af4d9c-4c6c-4265-a04b-7a2057193129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:45.411625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:45.425147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037649d5-514d-4620-a9de-c3a87cd19ada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:45.426287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037649d5-514d-4620-a9de-c3a87cd19ada service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:45.427690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037649d5-514d-4620-a9de-c3a87cd19ada service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:45.428184 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037649d5-514d-4620-a9de-c3a87cd19ada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:45.442805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037649d5-514d-4620-a9de-c3a87cd19ada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:45.444998 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037649d5-514d-4620-a9de-c3a87cd19ada service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:45.481815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 823/1648] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:45.482807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 826/1649] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:45.484032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037649d5-514d-4620-a9de-c3a87cd19ada service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:45.500559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037649d5-514d-4620-a9de-c3a87cd19ada service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:45.516253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a83a6-8fa4-4021-a5d5-e4029d60349f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:45.517127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a83a6-8fa4-4021-a5d5-e4029d60349f service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:45.517629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a83a6-8fa4-4021-a5d5-e4029d60349f service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:45.519211 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a83a6-8fa4-4021-a5d5-e4029d60349f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:45.534766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a83a6-8fa4-4021-a5d5-e4029d60349f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:45.536492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a83a6-8fa4-4021-a5d5-e4029d60349f service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:45.555253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 824/1650] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:46.266834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af4d9c-4c6c-4265-a04b-7a2057193129 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:46.283194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af4d9c-4c6c-4265-a04b-7a2057193129 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:46.293445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9602f-cd0b-498c-bce5-139cb651a77c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:46.295577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9602f-cd0b-498c-bce5-139cb651a77c service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:46.295870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9602f-cd0b-498c-bce5-139cb651a77c service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:46.297294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9602f-cd0b-498c-bce5-139cb651a77c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:46.324849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9602f-cd0b-498c-bce5-139cb651a77c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:46.328126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9602f-cd0b-498c-bce5-139cb651a77c service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:46.371004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 827/1651] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:46 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:47.785253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a83a6-8fa4-4021-a5d5-e4029d60349f service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:47.886591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9602f-cd0b-498c-bce5-139cb651a77c service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:48.039369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9602f-cd0b-498c-bce5-139cb651a77c service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:48.055054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee36005-5070-4377-a98d-50af0387fffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:48.055054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee36005-5070-4377-a98d-50af0387fffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:48.055054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee36005-5070-4377-a98d-50af0387fffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:48.055054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee36005-5070-4377-a98d-50af0387fffc admin admin] PATH_INFO: `/v3/users/5dd5eb59b183436099fb3d605ca421b1` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:48.056819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee36005-5070-4377-a98d-50af0387fffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd5eb59b183436099fb3d605ca421b1)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:48.063223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a83a6-8fa4-4021-a5d5-e4029d60349f service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:48.064402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee36005-5070-4377-a98d-50af0387fffc admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:48.096588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:48.097685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:48.098016 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:48.098299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] PATH_INFO: `/v3/users/707c9346e07d4fa29aabfab72b0f93db` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:48.105841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=707c9346e07d4fa29aabfab72b0f93db)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:48.107169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:14:48.107169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 04 09:14:48.107169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: Jul 04 09:14:48.108829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:48.211139 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1ee36005-5070-4377-a98d-50af0387fffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd5eb59b183436099fb3d605ca421b1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:48.212009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1ee36005-5070-4377-a98d-50af0387fffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd5eb59b183436099fb3d605ca421b1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:48.236389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1ee36005-5070-4377-a98d-50af0387fffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd5eb59b183436099fb3d605ca421b1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:48.246612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1ee36005-5070-4377-a98d-50af0387fffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd5eb59b183436099fb3d605ca421b1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:48.247511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1ee36005-5070-4377-a98d-50af0387fffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd5eb59b183436099fb3d605ca421b1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:48.284634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 828/1652] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:14:47 2022] DELETE /identity/v3/users/5dd5eb59b183436099fb3d605ca421b1 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:48.292675 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe994-6deb-4c8f-bb8f-c5017c5eaeed None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:48.293126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe994-6deb-4c8f-bb8f-c5017c5eaeed None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:48.293579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafe994-6deb-4c8f-bb8f-c5017c5eaeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:48.346884 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8dafe994-6deb-4c8f-bb8f-c5017c5eaeed None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:48.348766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8dafe994-6deb-4c8f-bb8f-c5017c5eaeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:48.378113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '707c9346e07d4fa29aabfab72b0f93db'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:48.380420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '707c9346e07d4fa29aabfab72b0f93db'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:48.384740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '707c9346e07d4fa29aabfab72b0f93db'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:48.405413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:48.415550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '707c9346e07d4fa29aabfab72b0f93db'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:48.451654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '707c9346e07d4fa29aabfab72b0f93db'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:48.513396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 829/1653] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:48.535483 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 825/1654] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:14:47 2022] DELETE /identity/v3/users/707c9346e07d4fa29aabfab72b0f93db => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:48.540642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065cb58-0e00-4247-a05b-5bb83e125a28 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:48.541303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065cb58-0e00-4247-a05b-5bb83e125a28 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:48.541797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065cb58-0e00-4247-a05b-5bb83e125a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:48.586731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5065cb58-0e00-4247-a05b-5bb83e125a28 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:48.590493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5065cb58-0e00-4247-a05b-5bb83e125a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:48.625257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:48.627022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:48.740217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:48.871743 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 830/1655] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:48.894531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4fd27-4359-4e6e-b6c4-8ef03e9ebd0f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:48.908037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc105b51-f8b1-476f-92bc-6dceaeed5e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:48.910462 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc105b51-f8b1-476f-92bc-6dceaeed5e35 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:48.913139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc105b51-f8b1-476f-92bc-6dceaeed5e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:48.913605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc105b51-f8b1-476f-92bc-6dceaeed5e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:48.946863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc105b51-f8b1-476f-92bc-6dceaeed5e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:48.953686 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc105b51-f8b1-476f-92bc-6dceaeed5e35 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:48.989185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 826/1656] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:48.991427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065cb58-0e00-4247-a05b-5bb83e125a28 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:49.011402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065cb58-0e00-4247-a05b-5bb83e125a28 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:49.025796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d92f41-98be-4b25-b527-be439b0145c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:49.026210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d92f41-98be-4b25-b527-be439b0145c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:49.026611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d92f41-98be-4b25-b527-be439b0145c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:49.026774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d92f41-98be-4b25-b527-be439b0145c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:49.042268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d92f41-98be-4b25-b527-be439b0145c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:49.044616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d92f41-98be-4b25-b527-be439b0145c3 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:49.096114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 831/1657] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:50.538084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc105b51-f8b1-476f-92bc-6dceaeed5e35 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:50.540192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fc105b51-f8b1-476f-92bc-6dceaeed5e35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:50.588766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc105b51-f8b1-476f-92bc-6dceaeed5e35 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:50.609596 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:50.610722 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:50.611546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:50.611950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] PATH_INFO: `/v3/projects/b811e2ed889f4b9c9a776e88971ef58a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:50.620182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b811e2ed889f4b9c9a776e88971ef58a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:50.627677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:50.760899 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b811e2ed889f4b9c9a776e88971ef58a', 'request_id': 'req-f857ac57-90e0-4e61-8281-938733c7b02c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:50.782332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b811e2ed889f4b9c9a776e88971ef58a', 'request_id': 'req-f857ac57-90e0-4e61-8281-938733c7b02c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:50.796394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b811e2ed889f4b9c9a776e88971ef58a', 'request_id': 'req-f857ac57-90e0-4e61-8281-938733c7b02c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:50.823723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] The token cache is being invalidate because project b811e2ed889f4b9c9a776e88971ef58a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:14:50.825982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:50.828075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:50.832955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 827/1658] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:14:50 2022] DELETE /identity/v3/projects/b811e2ed889f4b9c9a776e88971ef58a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:50.837276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d92f41-98be-4b25-b527-be439b0145c3 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:50.856548 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d92f41-98be-4b25-b527-be439b0145c3 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:51.039266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:51.039847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:51.040132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:51.040411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] PATH_INFO: `/v3/users/e122b65fab6e4758b95177c84626abef` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:51.044926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e122b65fab6e4758b95177c84626abef)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:51.049619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:51.185486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e122b65fab6e4758b95177c84626abef'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.186607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e122b65fab6e4758b95177c84626abef'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.208320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e122b65fab6e4758b95177c84626abef'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.211220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:51.211403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:51.213037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:51.247096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857ac57-90e0-4e61-8281-938733c7b02c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:51.247244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e122b65fab6e4758b95177c84626abef'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.254419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e122b65fab6e4758b95177c84626abef'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.257486 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:51.259063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:51.259565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:51.260001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] PATH_INFO: `/v3/projects/95d5c8e7c27d4e02972278937d08900c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:51.261736 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d5c8e7c27d4e02972278937d08900c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:51.267174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:51.345298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 832/1659] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:14:50 2022] DELETE /identity/v3/users/e122b65fab6e4758b95177c84626abef => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:51.352766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d5c8e7c27d4e02972278937d08900c', 'request_id': 'req-9188f3d9-79a7-4428-a43d-7a811d92f28b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.365146 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d5c8e7c27d4e02972278937d08900c', 'request_id': 'req-9188f3d9-79a7-4428-a43d-7a811d92f28b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.375385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d5c8e7c27d4e02972278937d08900c', 'request_id': 'req-9188f3d9-79a7-4428-a43d-7a811d92f28b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.437026 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] The token cache is being invalidate because project 95d5c8e7c27d4e02972278937d08900c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:14:51.438117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.444319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.449351 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 828/1660] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:14:51 2022] DELETE /identity/v3/projects/95d5c8e7c27d4e02972278937d08900c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:51.453453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:51.455116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:51.666788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfeea0-f0d5-4ea7-b7a8-dd4a853c1c7b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:51.689972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:51.696868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:51.701151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:51.702065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] PATH_INFO: `/v3/users/70f64e5ae02d44209def12327bcd6e2b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:51.704669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f64e5ae02d44209def12327bcd6e2b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:51.709993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:51.815279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f64e5ae02d44209def12327bcd6e2b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.816061 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f64e5ae02d44209def12327bcd6e2b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.838790 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f64e5ae02d44209def12327bcd6e2b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.853610 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f64e5ae02d44209def12327bcd6e2b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.855612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f64e5ae02d44209def12327bcd6e2b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:51.900490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 833/1661] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:14:51 2022] DELETE /identity/v3/users/70f64e5ae02d44209def12327bcd6e2b => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:52.391973 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:52.394084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:52.591924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:52.657369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:52.660870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:52.757703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188f3d9-79a7-4428-a43d-7a811d92f28b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:52.832626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:52.866209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d4d1d-fbc4-43f9-b055-69c2de62e905 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:52.879217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b937f8d-b268-449a-bfc6-f801fc364506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:52.880768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b937f8d-b268-449a-bfc6-f801fc364506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:52.881060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b937f8d-b268-449a-bfc6-f801fc364506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:52.881343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b937f8d-b268-449a-bfc6-f801fc364506 admin admin] PATH_INFO: `/v3/users/cc12f51acc3945d08bc1fb277f77ec67` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:52.884293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b937f8d-b268-449a-bfc6-f801fc364506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc12f51acc3945d08bc1fb277f77ec67)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:52.888532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b937f8d-b268-449a-bfc6-f801fc364506 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:52.978868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:52.980306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:52.980792 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:52.985092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] PATH_INFO: `/v3/projects/52aedf148cd943a5b8bbe70b19aeea49` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:52.990957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52aedf148cd943a5b8bbe70b19aeea49)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:52.993391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:53.009062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3b937f8d-b268-449a-bfc6-f801fc364506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc12f51acc3945d08bc1fb277f77ec67'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.010282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3b937f8d-b268-449a-bfc6-f801fc364506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc12f51acc3945d08bc1fb277f77ec67'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.047125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3b937f8d-b268-449a-bfc6-f801fc364506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc12f51acc3945d08bc1fb277f77ec67'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.083291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3b937f8d-b268-449a-bfc6-f801fc364506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc12f51acc3945d08bc1fb277f77ec67'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.085769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3b937f8d-b268-449a-bfc6-f801fc364506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc12f51acc3945d08bc1fb277f77ec67'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.132622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52aedf148cd943a5b8bbe70b19aeea49', 'request_id': 'req-52373cf2-2231-4e74-887f-83ac777231d6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.144560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 834/1662] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:14:52 2022] DELETE /identity/v3/users/cc12f51acc3945d08bc1fb277f77ec67 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:53.149912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0539c-f111-4c69-8d0f-6a233f6aac7c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:53.151127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52aedf148cd943a5b8bbe70b19aeea49', 'request_id': 'req-52373cf2-2231-4e74-887f-83ac777231d6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.151584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0539c-f111-4c69-8d0f-6a233f6aac7c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:53.152981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0539c-f111-4c69-8d0f-6a233f6aac7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:53.169465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52aedf148cd943a5b8bbe70b19aeea49', 'request_id': 'req-52373cf2-2231-4e74-887f-83ac777231d6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.196664 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-27e0539c-f111-4c69-8d0f-6a233f6aac7c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:53.198697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-27e0539c-f111-4c69-8d0f-6a233f6aac7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:53.226752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] The token cache is being invalidate because project 52aedf148cd943a5b8bbe70b19aeea49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:14:53.227319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.228357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-52373cf2-2231-4e74-887f-83ac777231d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.230231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 829/1663] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:14:52 2022] DELETE /identity/v3/projects/52aedf148cd943a5b8bbe70b19aeea49 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:53.236024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579be9c-384f-49ad-b0ff-a0fe95d5272b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:53.236563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579be9c-384f-49ad-b0ff-a0fe95d5272b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:53.237009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579be9c-384f-49ad-b0ff-a0fe95d5272b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:53.304636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7579be9c-384f-49ad-b0ff-a0fe95d5272b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:53.304636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7579be9c-384f-49ad-b0ff-a0fe95d5272b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:53.336143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 835/1664] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:53.385789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0539c-f111-4c69-8d0f-6a233f6aac7c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:53.387449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-27e0539c-f111-4c69-8d0f-6a233f6aac7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:53.509552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 830/1665] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:14:53.511718 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579be9c-384f-49ad-b0ff-a0fe95d5272b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:53.514939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7579be9c-384f-49ad-b0ff-a0fe95d5272b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:53.560831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579be9c-384f-49ad-b0ff-a0fe95d5272b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:53.575973 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:53.579111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:53.581773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:53.582315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] PATH_INFO: `/v3/projects/470b0b00efba49ecbab24afb7c2c5215` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:53.585624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=470b0b00efba49ecbab24afb7c2c5215)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:53.598915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:53.618395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:53.641862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0539c-f111-4c69-8d0f-6a233f6aac7c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:53.651020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104dee25-6965-4cfb-9f80-9e50aa4f5d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:53.651819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dee25-6965-4cfb-9f80-9e50aa4f5d9d service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:53.652363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dee25-6965-4cfb-9f80-9e50aa4f5d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:53.652731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dee25-6965-4cfb-9f80-9e50aa4f5d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:53.674467 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104dee25-6965-4cfb-9f80-9e50aa4f5d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:53.679094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104dee25-6965-4cfb-9f80-9e50aa4f5d9d service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:53.707004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 836/1666] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:53.707322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ea84a-8263-4fe7-a376-0e44b93af74d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:53.707865 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ea84a-8263-4fe7-a376-0e44b93af74d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:53.708223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ea84a-8263-4fe7-a376-0e44b93af74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:53.757915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-696ea84a-8263-4fe7-a376-0e44b93af74d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:53.759581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-696ea84a-8263-4fe7-a376-0e44b93af74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:53.796313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470b0b00efba49ecbab24afb7c2c5215', 'request_id': 'req-dd5b24b6-1622-46a8-9488-2c59934514ee'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.799147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:53.819843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470b0b00efba49ecbab24afb7c2c5215', 'request_id': 'req-dd5b24b6-1622-46a8-9488-2c59934514ee'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.839139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470b0b00efba49ecbab24afb7c2c5215', 'request_id': 'req-dd5b24b6-1622-46a8-9488-2c59934514ee'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.891082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] The token cache is being invalidate because project 470b0b00efba49ecbab24afb7c2c5215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:14:53.891386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.893017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dd5b24b6-1622-46a8-9488-2c59934514ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:53.914100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 831/1667] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:14:53 2022] DELETE /identity/v3/projects/470b0b00efba49ecbab24afb7c2c5215 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:53.918107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed047ed-ef1a-4c2f-9e7e-c7e04015ceda None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:53.921778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed047ed-ef1a-4c2f-9e7e-c7e04015ceda None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:53.922158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed047ed-ef1a-4c2f-9e7e-c7e04015ceda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:53.979364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3ed047ed-ef1a-4c2f-9e7e-c7e04015ceda None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:53.984458 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 837/1668] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:53.985515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ea84a-8263-4fe7-a376-0e44b93af74d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:53.990880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3ed047ed-ef1a-4c2f-9e7e-c7e04015ceda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:54.014889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ea84a-8263-4fe7-a376-0e44b93af74d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:54.020493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 832/1669] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:54.023923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce81928-d1e9-4917-b9c6-c6d59e30140c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:54.024630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce81928-d1e9-4917-b9c6-c6d59e30140c admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.024919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce81928-d1e9-4917-b9c6-c6d59e30140c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.025201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce81928-d1e9-4917-b9c6-c6d59e30140c admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.026999 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce81928-d1e9-4917-b9c6-c6d59e30140c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:14:54.027327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce81928-d1e9-4917-b9c6-c6d59e30140c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:54.030073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce81928-d1e9-4917-b9c6-c6d59e30140c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:54.044601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19699fb3-e3d9-44ee-84ec-905f87be382f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.048945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19699fb3-e3d9-44ee-84ec-905f87be382f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.049540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19699fb3-e3d9-44ee-84ec-905f87be382f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.056944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 838/1670] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:14:54.060136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce81928-d1e9-4917-b9c6-c6d59e30140c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:54.078666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce81928-d1e9-4917-b9c6-c6d59e30140c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:54.085376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba4e77-df0e-416e-adb6-93c595368768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:54.086205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba4e77-df0e-416e-adb6-93c595368768 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.086495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba4e77-df0e-416e-adb6-93c595368768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.086828 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba4e77-df0e-416e-adb6-93c595368768 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.087854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba4e77-df0e-416e-adb6-93c595368768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:54.090607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba4e77-df0e-416e-adb6-93c595368768 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:54.127933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-19699fb3-e3d9-44ee-84ec-905f87be382f None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:54.131642 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-19699fb3-e3d9-44ee-84ec-905f87be382f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:54.135563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 839/1671] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:54 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:54.142560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba4e77-df0e-416e-adb6-93c595368768 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:54.284379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 833/1672] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:54.288092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d7585-2786-40f4-a8c2-a1cef3ece937 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.288404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d7585-2786-40f4-a8c2-a1cef3ece937 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.288688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d7585-2786-40f4-a8c2-a1cef3ece937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.334734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba4e77-df0e-416e-adb6-93c595368768 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:54.351559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7dc16-12cb-400c-b499-07f41345f8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:54.352274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7dc16-12cb-400c-b499-07f41345f8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.352554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7dc16-12cb-400c-b499-07f41345f8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.352848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7dc16-12cb-400c-b499-07f41345f8b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.353561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7dc16-12cb-400c-b499-07f41345f8b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:54.356407 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7dc16-12cb-400c-b499-07f41345f8b7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:54.371504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b46d7585-2786-40f4-a8c2-a1cef3ece937 None None] MFA Rules not processed for user `1e916bf18efe421895c8d7b28fcd55d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:54.373922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b46d7585-2786-40f4-a8c2-a1cef3ece937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:54.397249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 840/1673] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:54 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:54.397249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7dc16-12cb-400c-b499-07f41345f8b7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:54.426439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7dc16-12cb-400c-b499-07f41345f8b7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:54.443749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250d9be-9c9a-4673-947a-df031fc1b18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:54.444556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250d9be-9c9a-4673-947a-df031fc1b18b admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.445148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250d9be-9c9a-4673-947a-df031fc1b18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.445796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250d9be-9c9a-4673-947a-df031fc1b18b admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.446538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250d9be-9c9a-4673-947a-df031fc1b18b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:14:54.446682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250d9be-9c9a-4673-947a-df031fc1b18b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:54.449559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250d9be-9c9a-4673-947a-df031fc1b18b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:54.487226 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 841/1674] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:14:54.488127 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250d9be-9c9a-4673-947a-df031fc1b18b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:54.516214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250d9be-9c9a-4673-947a-df031fc1b18b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:54.527636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc1d3b-9905-4f6c-a411-15647bf2d770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:54.528266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 834/1675] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:54.528756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc1d3b-9905-4f6c-a411-15647bf2d770 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.528756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc1d3b-9905-4f6c-a411-15647bf2d770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.528921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc1d3b-9905-4f6c-a411-15647bf2d770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.529578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc1d3b-9905-4f6c-a411-15647bf2d770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:54.530772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d7585-2786-40f4-a8c2-a1cef3ece937 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:54.531305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc1d3b-9905-4f6c-a411-15647bf2d770 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:54.546309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 842/1676] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:54.548170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0d1b-50f0-49e6-b2c2-53a2d4353238 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.548340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0d1b-50f0-49e6-b2c2-53a2d4353238 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.548617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f0d1b-50f0-49e6-b2c2-53a2d4353238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.554736 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d7585-2786-40f4-a8c2-a1cef3ece937 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:54.564050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe42fca-4f28-4c7f-81f4-481aa81f5a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:54.565318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe42fca-4f28-4c7f-81f4-481aa81f5a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.565950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe42fca-4f28-4c7f-81f4-481aa81f5a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.566404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe42fca-4f28-4c7f-81f4-481aa81f5a49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.568310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe42fca-4f28-4c7f-81f4-481aa81f5a49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:54.574591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe42fca-4f28-4c7f-81f4-481aa81f5a49 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:54.577596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a33f0d1b-50f0-49e6-b2c2-53a2d4353238 None None] MFA Rules not processed for user `1e916bf18efe421895c8d7b28fcd55d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:54.578924 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a33f0d1b-50f0-49e6-b2c2-53a2d4353238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:54.591118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 843/1677] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:54.592179 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f0d1b-50f0-49e6-b2c2-53a2d4353238 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:54.606104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:54.619178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f0d1b-50f0-49e6-b2c2-53a2d4353238 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:54.626885 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19b76c-b54e-40fb-8260-c297ab2f9b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:54.627810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b76c-b54e-40fb-8260-c297ab2f9b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.628196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b76c-b54e-40fb-8260-c297ab2f9b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.628591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b76c-b54e-40fb-8260-c297ab2f9b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.629438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19b76c-b54e-40fb-8260-c297ab2f9b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:54.632107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19b76c-b54e-40fb-8260-c297ab2f9b4d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:54.637023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 835/1678] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:54 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:54.638909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe42fca-4f28-4c7f-81f4-481aa81f5a49 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:54.641094 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bfe42fca-4f28-4c7f-81f4-481aa81f5a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:54.651615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 844/1679] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:54.651615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19b76c-b54e-40fb-8260-c297ab2f9b4d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:54.664378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19b76c-b54e-40fb-8260-c297ab2f9b4d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:54.806317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe42fca-4f28-4c7f-81f4-481aa81f5a49 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:54.813429 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bd79e-b59a-4079-983d-fa5ee2298455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:54.814448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bd79e-b59a-4079-983d-fa5ee2298455 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.814747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bd79e-b59a-4079-983d-fa5ee2298455 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.815030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bd79e-b59a-4079-983d-fa5ee2298455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.855807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:54.904335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bd79e-b59a-4079-983d-fa5ee2298455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:54.906978 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bd79e-b59a-4079-983d-fa5ee2298455 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:54.936609 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 836/1680] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:54.939583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bd79e-b59a-4079-983d-fa5ee2298455 service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:54.961062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b28c1-b62b-43b8-8f15-031712c52e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:54.961909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b28c1-b62b-43b8-8f15-031712c52e32 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:54.962226 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b28c1-b62b-43b8-8f15-031712c52e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:54.962648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b28c1-b62b-43b8-8f15-031712c52e32 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:54.963365 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b28c1-b62b-43b8-8f15-031712c52e32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:54.965810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b28c1-b62b-43b8-8f15-031712c52e32 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:55.004137 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 845/1681] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:54 2022] POST /identity/v3/users => generated 294 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:55.006808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b28c1-b62b-43b8-8f15-031712c52e32 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:55.022344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b28c1-b62b-43b8-8f15-031712c52e32 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:55.029303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08831bb-ec3f-4800-a697-6f7f10c16886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:55.031000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08831bb-ec3f-4800-a697-6f7f10c16886 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:55.031327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08831bb-ec3f-4800-a697-6f7f10c16886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:55.031609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08831bb-ec3f-4800-a697-6f7f10c16886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:55.032809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08831bb-ec3f-4800-a697-6f7f10c16886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:55.035783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08831bb-ec3f-4800-a697-6f7f10c16886 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:55.053306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 846/1682] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:55.054526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08831bb-ec3f-4800-a697-6f7f10c16886 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:55.063717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bd79e-b59a-4079-983d-fa5ee2298455 service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:55.066104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08831bb-ec3f-4800-a697-6f7f10c16886 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:55.074002 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ebae9-ac22-4b74-a4f8-1777a1031237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:55.075264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ebae9-ac22-4b74-a4f8-1777a1031237 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:55.075264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ebae9-ac22-4b74-a4f8-1777a1031237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:55.075457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ebae9-ac22-4b74-a4f8-1777a1031237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:55.076332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ebae9-ac22-4b74-a4f8-1777a1031237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:55.078262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20082c-55f1-4415-a9e8-3a23265c3e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:55.078772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ebae9-ac22-4b74-a4f8-1777a1031237 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:55.079378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20082c-55f1-4415-a9e8-3a23265c3e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:55.081210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20082c-55f1-4415-a9e8-3a23265c3e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:55.081541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20082c-55f1-4415-a9e8-3a23265c3e6c admin admin] PATH_INFO: `/v3/projects/3f00924a742e4c61bcfcce3b5410bcbe/users/6e729089f66946bb97fbfc0faf45463c/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:55.093775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 847/1683] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:55.098755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ebae9-ac22-4b74-a4f8-1777a1031237 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:55.120096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ebae9-ac22-4b74-a4f8-1777a1031237 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:55.130200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20082c-55f1-4415-a9e8-3a23265c3e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f00924a742e4c61bcfcce3b5410bcbe, user_id=6e729089f66946bb97fbfc0faf45463c, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:55.135061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20082c-55f1-4415-a9e8-3a23265c3e6c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:55.138765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83408e-d9b1-4abe-8801-9d8f3f1d1c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:55.142001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83408e-d9b1-4abe-8801-9d8f3f1d1c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:55.143683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83408e-d9b1-4abe-8801-9d8f3f1d1c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:55.145799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83408e-d9b1-4abe-8801-9d8f3f1d1c73 admin admin] PATH_INFO: `/v3/projects/d9c43aa6c3f64751a0e268bebe9625c3/users/50a04f09c095481f8a7c4008e7ead1d4/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:55.168003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 837/1684] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:54 2022] PUT /identity/v3/projects/3f00924a742e4c61bcfcce3b5410bcbe/users/6e729089f66946bb97fbfc0faf45463c/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:55.180065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77404741-8ddb-45c2-a85b-25f3743453ea None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:55.181303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77404741-8ddb-45c2-a85b-25f3743453ea None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:55.181558 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77404741-8ddb-45c2-a85b-25f3743453ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:55.203593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83408e-d9b1-4abe-8801-9d8f3f1d1c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c43aa6c3f64751a0e268bebe9625c3, user_id=50a04f09c095481f8a7c4008e7ead1d4, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:55.209252 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83408e-d9b1-4abe-8801-9d8f3f1d1c73 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:55.256355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 848/1685] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:55 2022] PUT /identity/v3/projects/d9c43aa6c3f64751a0e268bebe9625c3/users/50a04f09c095481f8a7c4008e7ead1d4/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:55.257299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83408e-d9b1-4abe-8801-9d8f3f1d1c73 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:55.264461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9b83408e-d9b1-4abe-8801-9d8f3f1d1c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:55.265212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-77404741-8ddb-45c2-a85b-25f3743453ea None None] MFA Rules not processed for user `6e729089f66946bb97fbfc0faf45463c`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:55.269102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-77404741-8ddb-45c2-a85b-25f3743453ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:55.282731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:55.473784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83408e-d9b1-4abe-8801-9d8f3f1d1c73 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:55.474406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 838/1686] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:55.476345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c1cfe-fcd3-4876-9476-07a43bd55374 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:55.476738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c1cfe-fcd3-4876-9476-07a43bd55374 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:55.477573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c1cfe-fcd3-4876-9476-07a43bd55374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:55.494079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:55.495043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:55.495398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:55.495678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] PATH_INFO: `/v3/projects/13eb5bc141c34f99ab7b090ff2ea1146` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:55.503970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13eb5bc141c34f99ab7b090ff2ea1146)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:55.505103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: WARNING py.warnings [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 04 09:14:55.505103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: warnings.warn(msg) Jul 04 09:14:55.505103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: Jul 04 09:14:55.506362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:55.524042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-091c1cfe-fcd3-4876-9476-07a43bd55374 None None] MFA Rules not processed for user `50a04f09c095481f8a7c4008e7ead1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:55.526999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-091c1cfe-fcd3-4876-9476-07a43bd55374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:55.673314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 839/1687] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:55.682166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c1cfe-fcd3-4876-9476-07a43bd55374 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:55.700055 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13eb5bc141c34f99ab7b090ff2ea1146', 'request_id': 'req-79b1ccab-3804-45ee-8abb-19818f63ee79'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:55.710350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c1cfe-fcd3-4876-9476-07a43bd55374 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:55.719538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13eb5bc141c34f99ab7b090ff2ea1146', 'request_id': 'req-79b1ccab-3804-45ee-8abb-19818f63ee79'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:55.742870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13eb5bc141c34f99ab7b090ff2ea1146', 'request_id': 'req-79b1ccab-3804-45ee-8abb-19818f63ee79'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:55.766641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:55.775827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] The token cache is being invalidate because project 13eb5bc141c34f99ab7b090ff2ea1146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:14:55.776113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:55.776868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:55.778633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 849/1688] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:14:55 2022] DELETE /identity/v3/projects/13eb5bc141c34f99ab7b090ff2ea1146 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:55.782680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:55.790183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:55.892701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac70d39-e0b3-4518-b6a5-21f89f0a3186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:55.893753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac70d39-e0b3-4518-b6a5-21f89f0a3186 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:55.894338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac70d39-e0b3-4518-b6a5-21f89f0a3186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:55.895054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac70d39-e0b3-4518-b6a5-21f89f0a3186 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:55.896210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac70d39-e0b3-4518-b6a5-21f89f0a3186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:55.898484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac70d39-e0b3-4518-b6a5-21f89f0a3186 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:55.901769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1ccab-3804-45ee-8abb-19818f63ee79 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:55.908024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50002fc-b72a-49b9-a76a-5fc6e0e7b59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:55.909647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50002fc-b72a-49b9-a76a-5fc6e0e7b59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:55.909965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50002fc-b72a-49b9-a76a-5fc6e0e7b59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:55.910599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50002fc-b72a-49b9-a76a-5fc6e0e7b59d admin admin] PATH_INFO: `/v3/users/ddea91415fb54f29a18db59ca41085aa` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:55.911819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50002fc-b72a-49b9-a76a-5fc6e0e7b59d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddea91415fb54f29a18db59ca41085aa)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:55.915141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50002fc-b72a-49b9-a76a-5fc6e0e7b59d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:55.965738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 840/1689] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:55 2022] POST /identity/v3/projects => generated 375 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:55.967874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac70d39-e0b3-4518-b6a5-21f89f0a3186 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:55.980011 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7ac70d39-e0b3-4518-b6a5-21f89f0a3186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:55.997738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b50002fc-b72a-49b9-a76a-5fc6e0e7b59d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddea91415fb54f29a18db59ca41085aa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:55.999214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b50002fc-b72a-49b9-a76a-5fc6e0e7b59d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddea91415fb54f29a18db59ca41085aa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:56.013310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:56.029096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b50002fc-b72a-49b9-a76a-5fc6e0e7b59d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddea91415fb54f29a18db59ca41085aa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:56.040387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b50002fc-b72a-49b9-a76a-5fc6e0e7b59d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddea91415fb54f29a18db59ca41085aa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:56.041315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b50002fc-b72a-49b9-a76a-5fc6e0e7b59d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddea91415fb54f29a18db59ca41085aa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:56.095658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 850/1690] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:14:55 2022] DELETE /identity/v3/users/ddea91415fb54f29a18db59ca41085aa => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:56.117371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b83a9c-c812-4cb8-baf8-6dcb79cd55fe None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:56.117841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b83a9c-c812-4cb8-baf8-6dcb79cd55fe None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:56.123912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b83a9c-c812-4cb8-baf8-6dcb79cd55fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:56.181635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac70d39-e0b3-4518-b6a5-21f89f0a3186 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:56.196319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-55b83a9c-c812-4cb8-baf8-6dcb79cd55fe None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:56.202274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-55b83a9c-c812-4cb8-baf8-6dcb79cd55fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:56.326348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd19a3-866f-4250-88df-4f97219998b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:56.327068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd19a3-866f-4250-88df-4f97219998b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:56.327360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd19a3-866f-4250-88df-4f97219998b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:56.327643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd19a3-866f-4250-88df-4f97219998b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:56.328375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd19a3-866f-4250-88df-4f97219998b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:56.335423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd19a3-866f-4250-88df-4f97219998b4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:56.364993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 851/1691] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:56.367060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc448e3-36de-45b0-9481-d39fb9f5be99 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:56.367614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc448e3-36de-45b0-9481-d39fb9f5be99 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:56.368065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc448e3-36de-45b0-9481-d39fb9f5be99 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:56.375537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 852/1692] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 4 09:14:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:14:56.376814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc448e3-36de-45b0-9481-d39fb9f5be99 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:56.379244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9dc448e3-36de-45b0-9481-d39fb9f5be99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:56.420198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc448e3-36de-45b0-9481-d39fb9f5be99 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:56.428721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 841/1693] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:55 2022] POST /identity/v3/projects => generated 385 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:56.433102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4e841-c485-4781-b6eb-9122cfe8aba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:56.435837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4e841-c485-4781-b6eb-9122cfe8aba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:56.436211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4e841-c485-4781-b6eb-9122cfe8aba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:56.438466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4e841-c485-4781-b6eb-9122cfe8aba7 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:56.440060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4e841-c485-4781-b6eb-9122cfe8aba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:56.440804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d4cc0-f610-43c9-a5b7-fc055277d0e6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:56.441257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d4cc0-f610-43c9-a5b7-fc055277d0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:56.442763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d4cc0-f610-43c9-a5b7-fc055277d0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:56.448311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4e841-c485-4781-b6eb-9122cfe8aba7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:56.487458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-296d4cc0-f610-43c9-a5b7-fc055277d0e6 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:56.491386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-296d4cc0-f610-43c9-a5b7-fc055277d0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:56.494852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 853/1694] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:56 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:56.497291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f1a31-cd0e-4f9d-b4a7-b4cf47c15a0e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:56.497640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f1a31-cd0e-4f9d-b4a7-b4cf47c15a0e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:56.498005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f1a31-cd0e-4f9d-b4a7-b4cf47c15a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:56.518084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:56.544630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2d8f1a31-cd0e-4f9d-b4a7-b4cf47c15a0e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:56.545877 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2d8f1a31-cd0e-4f9d-b4a7-b4cf47c15a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:56.632549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 842/1695] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:56.634418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb40d3-ecd8-4a55-8f26-2e8759157c54 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:56.634661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb40d3-ecd8-4a55-8f26-2e8759157c54 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:56.635123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb40d3-ecd8-4a55-8f26-2e8759157c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:56.700615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 854/1696] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:56.701270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f1a31-cd0e-4f9d-b4a7-b4cf47c15a0e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:56.703103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2d8f1a31-cd0e-4f9d-b4a7-b4cf47c15a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:56.722625 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-86cb40d3-ecd8-4a55-8f26-2e8759157c54 None None] MFA Rules not processed for user `98649aa38f764af0b41ff5594b94a989`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:56.723863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-86cb40d3-ecd8-4a55-8f26-2e8759157c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:56.726176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f1a31-cd0e-4f9d-b4a7-b4cf47c15a0e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:56.745601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89496ef0-1fb3-41bb-ba70-58cf848af67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:56.746329 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496ef0-1fb3-41bb-ba70-58cf848af67d admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:56.746612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496ef0-1fb3-41bb-ba70-58cf848af67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:56.746887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496ef0-1fb3-41bb-ba70-58cf848af67d admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:56.747605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89496ef0-1fb3-41bb-ba70-58cf848af67d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:56.749598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89496ef0-1fb3-41bb-ba70-58cf848af67d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:56.802896 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 855/1697] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:56 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:56.814804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89496ef0-1fb3-41bb-ba70-58cf848af67d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:56.826978 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:56.848284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89496ef0-1fb3-41bb-ba70-58cf848af67d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:56.866760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566b4a3-795c-4852-aba1-d37cbc2864cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:56.867577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566b4a3-795c-4852-aba1-d37cbc2864cf admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:56.867867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566b4a3-795c-4852-aba1-d37cbc2864cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:56.868146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566b4a3-795c-4852-aba1-d37cbc2864cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:56.868911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566b4a3-795c-4852-aba1-d37cbc2864cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:56.872231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566b4a3-795c-4852-aba1-d37cbc2864cf admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:56.912068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 843/1698] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:14:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:56.925846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb40d3-ecd8-4a55-8f26-2e8759157c54 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:56.927330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-86cb40d3-ecd8-4a55-8f26-2e8759157c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:56.941224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 856/1699] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:56.942052 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566b4a3-795c-4852-aba1-d37cbc2864cf admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:56.943779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb40d3-ecd8-4a55-8f26-2e8759157c54 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:56.955933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a37f61-6526-41a5-821a-76b915a2db1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:56.956626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a37f61-6526-41a5-821a-76b915a2db1d service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:56.956919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a37f61-6526-41a5-821a-76b915a2db1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:56.957196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a37f61-6526-41a5-821a-76b915a2db1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:56.960715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566b4a3-795c-4852-aba1-d37cbc2864cf admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:56.972852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a37f61-6526-41a5-821a-76b915a2db1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:56.976641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a37f61-6526-41a5-821a-76b915a2db1d service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:57.005090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95125e87-be24-455c-b677-f60049440c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:57.006014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95125e87-be24-455c-b677-f60049440c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:57.006405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95125e87-be24-455c-b677-f60049440c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:57.006788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95125e87-be24-455c-b677-f60049440c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:57.018340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95125e87-be24-455c-b677-f60049440c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:14:57.018340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95125e87-be24-455c-b677-f60049440c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:57.018340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95125e87-be24-455c-b677-f60049440c1d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:57.050041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 844/1700] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:57.052147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a37f61-6526-41a5-821a-76b915a2db1d service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:57.056638 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 857/1701] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:14:57.070148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95125e87-be24-455c-b677-f60049440c1d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:57.084877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a37f61-6526-41a5-821a-76b915a2db1d service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:57.101366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc8ea-b5ab-4d31-ac44-f944c2426596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:57.102474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc8ea-b5ab-4d31-ac44-f944c2426596 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:57.102972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc8ea-b5ab-4d31-ac44-f944c2426596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:57.103594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc8ea-b5ab-4d31-ac44-f944c2426596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:57.104600 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bc8ea-b5ab-4d31-ac44-f944c2426596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:57.108809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bc8ea-b5ab-4d31-ac44-f944c2426596 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:57.109556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95125e87-be24-455c-b677-f60049440c1d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:57.119580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdeaf33-c05f-45de-9a35-0fa52484e7a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:57.122655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeaf33-c05f-45de-9a35-0fa52484e7a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:57.122965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeaf33-c05f-45de-9a35-0fa52484e7a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:57.129187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeaf33-c05f-45de-9a35-0fa52484e7a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:57.165700 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 845/1702] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:57.171268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc8ea-b5ab-4d31-ac44-f944c2426596 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:57.173498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdeaf33-c05f-45de-9a35-0fa52484e7a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:57.179844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdeaf33-c05f-45de-9a35-0fa52484e7a8 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:57.206206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc8ea-b5ab-4d31-ac44-f944c2426596 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:57.220916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192394a-b4a0-475a-9f76-d221ce250b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:57.222129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192394a-b4a0-475a-9f76-d221ce250b76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:57.222490 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192394a-b4a0-475a-9f76-d221ce250b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:57.224038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192394a-b4a0-475a-9f76-d221ce250b76 admin admin] PATH_INFO: `/v3/projects/bd8f65b210554291acf018cabc185fe6/users/3121c7e90d3643b7af969170ef21b73b/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:57.248308 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 858/1703] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:57.249010 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdeaf33-c05f-45de-9a35-0fa52484e7a8 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:57.275117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192394a-b4a0-475a-9f76-d221ce250b76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd8f65b210554291acf018cabc185fe6, user_id=3121c7e90d3643b7af969170ef21b73b, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:57.283905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192394a-b4a0-475a-9f76-d221ce250b76 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:57.297394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdeaf33-c05f-45de-9a35-0fa52484e7a8 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:57.306189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b204ed2-de41-49d6-9ddc-30a56f6c7e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:57.307453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204ed2-de41-49d6-9ddc-30a56f6c7e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:57.307589 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204ed2-de41-49d6-9ddc-30a56f6c7e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:57.308047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204ed2-de41-49d6-9ddc-30a56f6c7e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:57.308848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b204ed2-de41-49d6-9ddc-30a56f6c7e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:57.312213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 846/1704] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:57 2022] PUT /identity/v3/projects/bd8f65b210554291acf018cabc185fe6/users/3121c7e90d3643b7af969170ef21b73b/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:57.314946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b204ed2-de41-49d6-9ddc-30a56f6c7e1b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:57.316203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192394a-b4a0-475a-9f76-d221ce250b76 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:57.385809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 859/1705] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:14:57 2022] POST /identity/v3/projects => generated 405 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:57.400704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b204ed2-de41-49d6-9ddc-30a56f6c7e1b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:57.529805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192394a-b4a0-475a-9f76-d221ce250b76 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:57.580056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b204ed2-de41-49d6-9ddc-30a56f6c7e1b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:57.593091 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f6201-6e63-42a8-a1ad-f4d1f180b9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:57.594131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f6201-6e63-42a8-a1ad-f4d1f180b9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:57.595706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f6201-6e63-42a8-a1ad-f4d1f180b9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:57.596826 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f6201-6e63-42a8-a1ad-f4d1f180b9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:57.598035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f6201-6e63-42a8-a1ad-f4d1f180b9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:57.606119 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f6201-6e63-42a8-a1ad-f4d1f180b9d0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:57.637406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 860/1706] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:57.639768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f6201-6e63-42a8-a1ad-f4d1f180b9d0 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:57.661945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f6201-6e63-42a8-a1ad-f4d1f180b9d0 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:57.674136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cac33-1e5b-4e69-87ad-33c821abfdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:57.675380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cac33-1e5b-4e69-87ad-33c821abfdd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:57.675930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cac33-1e5b-4e69-87ad-33c821abfdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:57.677288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cac33-1e5b-4e69-87ad-33c821abfdd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:57.678920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cac33-1e5b-4e69-87ad-33c821abfdd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:57.685471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1cac33-1e5b-4e69-87ad-33c821abfdd0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:57.702053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3eca8b-c08a-415f-a515-dd090e9339ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:57.703980 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eca8b-c08a-415f-a515-dd090e9339ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:57.704800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eca8b-c08a-415f-a515-dd090e9339ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:57.705909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eca8b-c08a-415f-a515-dd090e9339ef admin admin] PATH_INFO: `/v3/projects/5daa0d76de7c46d49d865277904e98fa/users/0caad1fac8b64ba0971a4a184621af42/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:57.738209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 861/1707] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:14:57 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:14:57.740145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc64654-87e5-49b2-9716-b41e5ff3c439 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:57.740438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc64654-87e5-49b2-9716-b41e5ff3c439 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:57.740867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc64654-87e5-49b2-9716-b41e5ff3c439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:57.751318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3eca8b-c08a-415f-a515-dd090e9339ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5daa0d76de7c46d49d865277904e98fa, user_id=0caad1fac8b64ba0971a4a184621af42, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:57.757283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3eca8b-c08a-415f-a515-dd090e9339ef admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:57.761139 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:57.795528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 847/1708] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:57 2022] PUT /identity/v3/projects/5daa0d76de7c46d49d865277904e98fa/users/0caad1fac8b64ba0971a4a184621af42/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:57.799661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3eca8b-c08a-415f-a515-dd090e9339ef admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:57.822620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-bcc64654-87e5-49b2-9716-b41e5ff3c439 None None] MFA Rules not processed for user `cb99f1e73a214b4ab32102a9ab5670fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:57.828455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bcc64654-87e5-49b2-9716-b41e5ff3c439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:57.933136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:57.994105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 862/1709] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:57.994105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc64654-87e5-49b2-9716-b41e5ff3c439 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:57.994874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3eca8b-c08a-415f-a515-dd090e9339ef admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:58.004294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0517c4-e1ba-4365-808a-84ee6d11425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:58.005236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0517c4-e1ba-4365-808a-84ee6d11425c admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:58.005517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0517c4-e1ba-4365-808a-84ee6d11425c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:58.005792 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0517c4-e1ba-4365-808a-84ee6d11425c admin admin] PATH_INFO: `/v3/projects/bd8f65b210554291acf018cabc185fe6/users/3121c7e90d3643b7af969170ef21b73b/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:58.009091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0517c4-e1ba-4365-808a-84ee6d11425c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd8f65b210554291acf018cabc185fe6, user_id=3121c7e90d3643b7af969170ef21b73b, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:58.016473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0517c4-e1ba-4365-808a-84ee6d11425c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:58.025559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc64654-87e5-49b2-9716-b41e5ff3c439 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:58.037607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f1757-d8c3-49eb-bd7f-03684e8374f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:58.038944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f1757-d8c3-49eb-bd7f-03684e8374f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:58.040032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f1757-d8c3-49eb-bd7f-03684e8374f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:58.041294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f1757-d8c3-49eb-bd7f-03684e8374f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:58.051126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f1757-d8c3-49eb-bd7f-03684e8374f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:58.053291 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 848/1710] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:57 2022] PUT /identity/v3/projects/bd8f65b210554291acf018cabc185fe6/users/3121c7e90d3643b7af969170ef21b73b/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:58.054376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0517c4-e1ba-4365-808a-84ee6d11425c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:58.056174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f1757-d8c3-49eb-bd7f-03684e8374f9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:58.096015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 863/1711] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:58.101542 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc9ef6-e28c-4a3b-b04b-f7b537a8658f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:58.101946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc9ef6-e28c-4a3b-b04b-f7b537a8658f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:58.102920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc9ef6-e28c-4a3b-b04b-f7b537a8658f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:58.209294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-31fc9ef6-e28c-4a3b-b04b-f7b537a8658f None None] MFA Rules not processed for user `cb99f1e73a214b4ab32102a9ab5670fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:58.210871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-31fc9ef6-e28c-4a3b-b04b-f7b537a8658f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:58.326554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0517c4-e1ba-4365-808a-84ee6d11425c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:58.345784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2f698-e31a-43d4-8d1e-cb52a893721e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:58.346570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2f698-e31a-43d4-8d1e-cb52a893721e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:58.346794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2f698-e31a-43d4-8d1e-cb52a893721e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:58.347638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2f698-e31a-43d4-8d1e-cb52a893721e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:58.348461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2f698-e31a-43d4-8d1e-cb52a893721e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:58.350144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2f698-e31a-43d4-8d1e-cb52a893721e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:58.388670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 864/1712] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:58.390235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716f6a9-77b9-41bf-a697-79661ebc5ea1 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:58.390476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716f6a9-77b9-41bf-a697-79661ebc5ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:58.390757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716f6a9-77b9-41bf-a697-79661ebc5ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:58.400266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 849/1713] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:58.401612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2f698-e31a-43d4-8d1e-cb52a893721e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:58.423735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2f698-e31a-43d4-8d1e-cb52a893721e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:58.436124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1772715-ce95-4c2a-a534-e13379bfa2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:58.438044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1772715-ce95-4c2a-a534-e13379bfa2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:58.438519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1772715-ce95-4c2a-a534-e13379bfa2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:58.439949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1772715-ce95-4c2a-a534-e13379bfa2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:58.442333 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1772715-ce95-4c2a-a534-e13379bfa2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:58.444970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1772715-ce95-4c2a-a534-e13379bfa2b3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:58.449513 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c716f6a9-77b9-41bf-a697-79661ebc5ea1 None None] MFA Rules not processed for user `3121c7e90d3643b7af969170ef21b73b`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:58.451959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c716f6a9-77b9-41bf-a697-79661ebc5ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:58.459059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 850/1714] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:14:58.460270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1772715-ce95-4c2a-a534-e13379bfa2b3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:58.464886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f1772715-ce95-4c2a-a534-e13379bfa2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:58.496289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1772715-ce95-4c2a-a534-e13379bfa2b3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:58.511824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:58.513119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:58.513365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:58.513781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] PATH_INFO: `/v3/projects/80a6421aa9a94f0badd40600792880fd` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:58.516641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a6421aa9a94f0badd40600792880fd)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:58.522917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:58.644154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 865/1715] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:58.648545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716f6a9-77b9-41bf-a697-79661ebc5ea1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:58.664213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716f6a9-77b9-41bf-a697-79661ebc5ea1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:58.675384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:58.718507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a6421aa9a94f0badd40600792880fd', 'request_id': 'req-60a539c5-5ae5-48ea-902f-3537f8efb840'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:58.732052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a6421aa9a94f0badd40600792880fd', 'request_id': 'req-60a539c5-5ae5-48ea-902f-3537f8efb840'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:58.753796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a6421aa9a94f0badd40600792880fd', 'request_id': 'req-60a539c5-5ae5-48ea-902f-3537f8efb840'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:58.779766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d981ce-4b71-4aed-b2ed-2a1ce8099336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:58.781639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d981ce-4b71-4aed-b2ed-2a1ce8099336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:58.781962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d981ce-4b71-4aed-b2ed-2a1ce8099336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:58.782483 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d981ce-4b71-4aed-b2ed-2a1ce8099336 admin admin] PATH_INFO: `/v3/projects/5daa0d76de7c46d49d865277904e98fa/users/0caad1fac8b64ba0971a4a184621af42/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:58.791438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] The token cache is being invalidate because project 80a6421aa9a94f0badd40600792880fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:14:58.791438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:58.793841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d981ce-4b71-4aed-b2ed-2a1ce8099336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5daa0d76de7c46d49d865277904e98fa, user_id=0caad1fac8b64ba0971a4a184621af42, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:58.796337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:14:58.803621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d981ce-4b71-4aed-b2ed-2a1ce8099336 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:58.806505 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 851/1716] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:14:58 2022] DELETE /identity/v3/projects/80a6421aa9a94f0badd40600792880fd => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:58.807408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:58.810487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:58.844663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 866/1717] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:58 2022] PUT /identity/v3/projects/5daa0d76de7c46d49d865277904e98fa/users/0caad1fac8b64ba0971a4a184621af42/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:58.845707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d981ce-4b71-4aed-b2ed-2a1ce8099336 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:58.847464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a5d981ce-4b71-4aed-b2ed-2a1ce8099336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:58.889582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:58.972907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d981ce-4b71-4aed-b2ed-2a1ce8099336 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:58.992020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28726570-3e88-47c3-81aa-eb9928a3fece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:58.992862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28726570-3e88-47c3-81aa-eb9928a3fece admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:58.993144 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28726570-3e88-47c3-81aa-eb9928a3fece admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:58.993351 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a539c5-5ae5-48ea-902f-3537f8efb840 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:58.993458 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28726570-3e88-47c3-81aa-eb9928a3fece admin admin] PATH_INFO: `/v3/projects/00794e1916234e4b8e20037eec608eb0/users/0208b6c80c0f4e4e9e093286b0843732/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:59.026721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28726570-3e88-47c3-81aa-eb9928a3fece admin admin] RBAC: Authorizing `identity:create_grant(project_id=00794e1916234e4b8e20037eec608eb0, user_id=0208b6c80c0f4e4e9e093286b0843732, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:59.029299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28726570-3e88-47c3-81aa-eb9928a3fece admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:59.068747 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 867/1718] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:14:58 2022] PUT /identity/v3/projects/00794e1916234e4b8e20037eec608eb0/users/0208b6c80c0f4e4e9e093286b0843732/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:14:59.070096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225551c-63bc-42ad-9cd0-6af62eb92bfc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:59.070537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225551c-63bc-42ad-9cd0-6af62eb92bfc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:59.070642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225551c-63bc-42ad-9cd0-6af62eb92bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:59.119030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e225551c-63bc-42ad-9cd0-6af62eb92bfc None None] MFA Rules not processed for user `43bcd8fb25604011bfd1df823ea4ddad`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:59.122104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e225551c-63bc-42ad-9cd0-6af62eb92bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:59.177683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febd752-92d0-437c-bca3-cc1dd6bf6fb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:59.178515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd752-92d0-437c-bca3-cc1dd6bf6fb2 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:59.178801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd752-92d0-437c-bca3-cc1dd6bf6fb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:59.179162 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd752-92d0-437c-bca3-cc1dd6bf6fb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:59.181568 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7febd752-92d0-437c-bca3-cc1dd6bf6fb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:59.243108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 868/1719] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:59.244496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225551c-63bc-42ad-9cd0-6af62eb92bfc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:59.246543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e225551c-63bc-42ad-9cd0-6af62eb92bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:59.312676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febd752-92d0-437c-bca3-cc1dd6bf6fb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:59.313263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:59.314978 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febd752-92d0-437c-bca3-cc1dd6bf6fb2 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:59.340632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 852/1720] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:59.342433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d89c-2705-48cb-9074-c88175868508 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:59.344591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d89c-2705-48cb-9074-c88175868508 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:59.345270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d89c-2705-48cb-9074-c88175868508 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:59.351876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 853/1721] 10.222.0.26 () {54 vars in 960 bytes} [Mon Jul 4 09:14:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:14:59.353342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07483f-f888-4ef7-a40f-bc84e23d67c5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:59.354053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07483f-f888-4ef7-a40f-bc84e23d67c5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:59.354569 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07483f-f888-4ef7-a40f-bc84e23d67c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:59.404752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1c07483f-f888-4ef7-a40f-bc84e23d67c5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:59.406261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1c07483f-f888-4ef7-a40f-bc84e23d67c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:59.439340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225551c-63bc-42ad-9cd0-6af62eb92bfc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:14:59.445975 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23520e6-eb04-4ddc-a2c2-4547fd2df710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:14:59.446989 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23520e6-eb04-4ddc-a2c2-4547fd2df710 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:59.447529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23520e6-eb04-4ddc-a2c2-4547fd2df710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:59.449412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23520e6-eb04-4ddc-a2c2-4547fd2df710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:59.465084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b23520e6-eb04-4ddc-a2c2-4547fd2df710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:59.531937 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 854/1722] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:59.534637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd8bff-f3a9-4aaf-bebb-95c3920e039f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:59.534934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd8bff-f3a9-4aaf-bebb-95c3920e039f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:59.535215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd8bff-f3a9-4aaf-bebb-95c3920e039f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:59.569218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:14:59.588447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b6cd8bff-f3a9-4aaf-bebb-95c3920e039f None None] MFA Rules not processed for user `0caad1fac8b64ba0971a4a184621af42`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:59.589915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b6cd8bff-f3a9-4aaf-bebb-95c3920e039f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:59.592847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23520e6-eb04-4ddc-a2c2-4547fd2df710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:14:59.594891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23520e6-eb04-4ddc-a2c2-4547fd2df710 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:14:59.626340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 869/1723] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:14:59.630558 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb824486-f5b0-45d5-9d07-92de8a05cf22 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:59.630850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb824486-f5b0-45d5-9d07-92de8a05cf22 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:59.631129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb824486-f5b0-45d5-9d07-92de8a05cf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:59.696178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-bb824486-f5b0-45d5-9d07-92de8a05cf22 None None] MFA Rules not processed for user `1e916bf18efe421895c8d7b28fcd55d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:59.697427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bb824486-f5b0-45d5-9d07-92de8a05cf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:59.718058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 855/1724] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:59.719924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c7980-6281-45ce-9d46-b2e2283d59a4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:14:59.720039 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c7980-6281-45ce-9d46-b2e2283d59a4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:14:59.720379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c7980-6281-45ce-9d46-b2e2283d59a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:14:59.798490 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-eb5c7980-6281-45ce-9d46-b2e2283d59a4 None None] MFA Rules not processed for user `0208b6c80c0f4e4e9e093286b0843732`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:14:59.803112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-eb5c7980-6281-45ce-9d46-b2e2283d59a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:59.825866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 870/1725] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:14:59.826866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb824486-f5b0-45d5-9d07-92de8a05cf22 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:14:59.834789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bb824486-f5b0-45d5-9d07-92de8a05cf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:14:59.996374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 856/1726] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:14:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:00.003370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7983bc-5e5a-465c-9cc4-2f2a21772606 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:00.003663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7983bc-5e5a-465c-9cc4-2f2a21772606 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:00.006268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7983bc-5e5a-465c-9cc4-2f2a21772606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:00.031263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb824486-f5b0-45d5-9d07-92de8a05cf22 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:00.053560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ee7983bc-5e5a-465c-9cc4-2f2a21772606 None None] MFA Rules not processed for user `aff2305249a242639e1415094bf9df33`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:00.054580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51ab69-a334-440f-aa4b-90f436c13682 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:00.055291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ab69-a334-440f-aa4b-90f436c13682 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:00.055719 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ab69-a334-440f-aa4b-90f436c13682 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:00.056000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ab69-a334-440f-aa4b-90f436c13682 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:00.059422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ee7983bc-5e5a-465c-9cc4-2f2a21772606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:00.098708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51ab69-a334-440f-aa4b-90f436c13682 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:00.100385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51ab69-a334-440f-aa4b-90f436c13682 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:00.167913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 871/1727] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:14:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:00.169255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd420f30-fd1f-4975-93ed-83319d2a4998 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:00.169561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd420f30-fd1f-4975-93ed-83319d2a4998 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:00.169838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd420f30-fd1f-4975-93ed-83319d2a4998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:00.197705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:00.218410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-fd420f30-fd1f-4975-93ed-83319d2a4998 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:00.219731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fd420f30-fd1f-4975-93ed-83319d2a4998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:00.262393 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 872/1728] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:00.263366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd420f30-fd1f-4975-93ed-83319d2a4998 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:00.267118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fd420f30-fd1f-4975-93ed-83319d2a4998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:00.293763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd420f30-fd1f-4975-93ed-83319d2a4998 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:00.316343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 857/1729] 10.222.0.26 () {58 vars in 1057 bytes} [Mon Jul 4 09:14:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:00.317743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bd48c-d4c3-4c36-9ccc-c1311bff10e7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:00.318070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bd48c-d4c3-4c36-9ccc-c1311bff10e7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:00.318341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bd48c-d4c3-4c36-9ccc-c1311bff10e7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:00.321816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3c10b-597f-4562-975f-049c8236c329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:00.322499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3c10b-597f-4562-975f-049c8236c329 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:00.322789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3c10b-597f-4562-975f-049c8236c329 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:00.323074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3c10b-597f-4562-975f-049c8236c329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:00.323831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 858/1730] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 4 09:15:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:15:00.324622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bd48c-d4c3-4c36-9ccc-c1311bff10e7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:00.341348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3c10b-597f-4562-975f-049c8236c329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:00.342840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3c10b-597f-4562-975f-049c8236c329 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:00.355855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bd48c-d4c3-4c36-9ccc-c1311bff10e7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:00.365452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 873/1731] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:00.366153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3c10b-597f-4562-975f-049c8236c329 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:00.367788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378ccb8-5855-47af-b36e-300ac6aa88a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:00.368583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378ccb8-5855-47af-b36e-300ac6aa88a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:00.368760 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378ccb8-5855-47af-b36e-300ac6aa88a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:00.369007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378ccb8-5855-47af-b36e-300ac6aa88a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:00.369803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378ccb8-5855-47af-b36e-300ac6aa88a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:00.371717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378ccb8-5855-47af-b36e-300ac6aa88a4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:00.393220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3c10b-597f-4562-975f-049c8236c329 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:00.401958 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ed236-03f9-47c9-bc06-36a5e4beb1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:00.403084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ed236-03f9-47c9-bc06-36a5e4beb1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:00.403364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ed236-03f9-47c9-bc06-36a5e4beb1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:00.403645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ed236-03f9-47c9-bc06-36a5e4beb1e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:00.404419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ed236-03f9-47c9-bc06-36a5e4beb1e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:00.404725 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ed236-03f9-47c9-bc06-36a5e4beb1e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:00.408050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:00.409044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ed236-03f9-47c9-bc06-36a5e4beb1e5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:00.424303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 859/1732] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:00 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:00.428023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378ccb8-5855-47af-b36e-300ac6aa88a4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:00.449185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 874/1733] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:00.463049 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a46e5-33e0-4b89-a4ab-5a4905fb4a89 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:00.463342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a46e5-33e0-4b89-a4ab-5a4905fb4a89 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:00.463626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a46e5-33e0-4b89-a4ab-5a4905fb4a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:00.510637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-789a46e5-33e0-4b89-a4ab-5a4905fb4a89 None None] MFA Rules not processed for user `98649aa38f764af0b41ff5594b94a989`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:00.512201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-789a46e5-33e0-4b89-a4ab-5a4905fb4a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:00.623224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378ccb8-5855-47af-b36e-300ac6aa88a4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:00.634849 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9ec11-d893-4d5a-a5df-bec46ac14500 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:00.635563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9ec11-d893-4d5a-a5df-bec46ac14500 service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:00.635924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9ec11-d893-4d5a-a5df-bec46ac14500 service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:00.636247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9ec11-d893-4d5a-a5df-bec46ac14500 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:00.642550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-73d9ec11-d893-4d5a-a5df-bec46ac14500 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:00.657289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 875/1734] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:15:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:00.658856 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a46e5-33e0-4b89-a4ab-5a4905fb4a89 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:00.666040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9ec11-d893-4d5a-a5df-bec46ac14500 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:00.676379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9ec11-d893-4d5a-a5df-bec46ac14500 service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:00.707472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 860/1735] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:00.708070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9ec11-d893-4d5a-a5df-bec46ac14500 service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:00.835235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a46e5-33e0-4b89-a4ab-5a4905fb4a89 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:00.851535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ccbee-fd75-477a-bb6d-617b974179c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:00.852215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ccbee-fd75-477a-bb6d-617b974179c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:00.852615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ccbee-fd75-477a-bb6d-617b974179c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:00.852987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ccbee-fd75-477a-bb6d-617b974179c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:00.864092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ccbee-fd75-477a-bb6d-617b974179c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:00.867302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ccbee-fd75-477a-bb6d-617b974179c4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:00.924058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 876/1736] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:00 2022] POST /identity/v3/users => generated 308 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:00.928508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ccbee-fd75-477a-bb6d-617b974179c4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:00.931688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9ec11-d893-4d5a-a5df-bec46ac14500 service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:00.936019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-902ccbee-fd75-477a-bb6d-617b974179c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:00.949309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ba5ab-d35f-406b-bda5-83b43fdc29df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:00.954584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ba5ab-d35f-406b-bda5-83b43fdc29df admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:00.954912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ba5ab-d35f-406b-bda5-83b43fdc29df admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:00.955202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ba5ab-d35f-406b-bda5-83b43fdc29df admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:00.960503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ba5ab-d35f-406b-bda5-83b43fdc29df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:00.965301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ba5ab-d35f-406b-bda5-83b43fdc29df admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:00.978680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ccbee-fd75-477a-bb6d-617b974179c4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:01.002951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:01.009358 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:01.011233 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:01.011654 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] PATH_INFO: `/v3/users/50a04f09c095481f8a7c4008e7ead1d4` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:01.015629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a04f09c095481f8a7c4008e7ead1d4)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:01.017715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:01.054068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 861/1737] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:00 2022] POST /identity/v3/projects => generated 373 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:01.055088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ba5ab-d35f-406b-bda5-83b43fdc29df admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:01.174392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a04f09c095481f8a7c4008e7ead1d4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:01.175273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a04f09c095481f8a7c4008e7ead1d4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:01.200696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a04f09c095481f8a7c4008e7ead1d4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:01.217419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a04f09c095481f8a7c4008e7ead1d4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:01.218838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a04f09c095481f8a7c4008e7ead1d4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:01.224549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:01.262776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 877/1738] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:00 2022] DELETE /identity/v3/users/50a04f09c095481f8a7c4008e7ead1d4 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:01.263323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ba5ab-d35f-406b-bda5-83b43fdc29df admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:01.263977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:01.265494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:01.327632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:01.470539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f33146-b999-40c3-9284-3584fb1a8941 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:01.481380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192a47f-d9e0-42de-8f21-116a27a07ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:01.482452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a47f-d9e0-42de-8f21-116a27a07ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:01.482882 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a47f-d9e0-42de-8f21-116a27a07ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:01.483378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a47f-d9e0-42de-8f21-116a27a07ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:01.484172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192a47f-d9e0-42de-8f21-116a27a07ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:01.489416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192a47f-d9e0-42de-8f21-116a27a07ce6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:01.497157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564e476-2e2d-4563-99d4-545f553925f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:01.498394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564e476-2e2d-4563-99d4-545f553925f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:01.498844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564e476-2e2d-4563-99d4-545f553925f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:01.501000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564e476-2e2d-4563-99d4-545f553925f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:01.502334 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564e476-2e2d-4563-99d4-545f553925f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:01.508981 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564e476-2e2d-4563-99d4-545f553925f3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:01.530574 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 878/1739] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:01 2022] POST /identity/v3/users => generated 287 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:01.532448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192a47f-d9e0-42de-8f21-116a27a07ce6 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:01.534874 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6192a47f-d9e0-42de-8f21-116a27a07ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:01.560103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 862/1740] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:01.561512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7906e-27ab-490f-9e14-16607be50230 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:01.561852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7906e-27ab-490f-9e14-16607be50230 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:01.562163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7906e-27ab-490f-9e14-16607be50230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:01.612060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f7b7906e-27ab-490f-9e14-16607be50230 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:01.619112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f7b7906e-27ab-490f-9e14-16607be50230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:01.638361 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 863/1741] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:01.639677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7906e-27ab-490f-9e14-16607be50230 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:01.658667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7906e-27ab-490f-9e14-16607be50230 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:01.664991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61f343-6e9e-4d86-9fcb-cea68a8fad9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:01.666070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61f343-6e9e-4d86-9fcb-cea68a8fad9e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:01.666561 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61f343-6e9e-4d86-9fcb-cea68a8fad9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:01.667110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61f343-6e9e-4d86-9fcb-cea68a8fad9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:01.669427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61f343-6e9e-4d86-9fcb-cea68a8fad9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:01.671709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61f343-6e9e-4d86-9fcb-cea68a8fad9e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:01.674141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192a47f-d9e0-42de-8f21-116a27a07ce6 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:01.681341 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81427700-da25-45df-a6b1-a190c72c1a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:01.682075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81427700-da25-45df-a6b1-a190c72c1a32 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:01.682361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81427700-da25-45df-a6b1-a190c72c1a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:01.682688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81427700-da25-45df-a6b1-a190c72c1a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:01.691745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 864/1742] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:01.691927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-81427700-da25-45df-a6b1-a190c72c1a32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:01.693977 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61f343-6e9e-4d86-9fcb-cea68a8fad9e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:01.696513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ab61f343-6e9e-4d86-9fcb-cea68a8fad9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:01.719693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61f343-6e9e-4d86-9fcb-cea68a8fad9e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:01.725972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81427700-da25-45df-a6b1-a190c72c1a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:01.728129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81427700-da25-45df-a6b1-a190c72c1a32 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:01.728695 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ccfad-f325-4ce1-a29b-3f550c7d9393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:01.729963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ccfad-f325-4ce1-a29b-3f550c7d9393 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:01.730482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ccfad-f325-4ce1-a29b-3f550c7d9393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:01.730990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ccfad-f325-4ce1-a29b-3f550c7d9393 admin admin] PATH_INFO: `/v3/projects/6883dcfdc0954c199771ff4aca551997/users/b21a0291c3c8483db4dd56b2547c26b1/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:01.754826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ccfad-f325-4ce1-a29b-3f550c7d9393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6883dcfdc0954c199771ff4aca551997, user_id=b21a0291c3c8483db4dd56b2547c26b1, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:01.759273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ccfad-f325-4ce1-a29b-3f550c7d9393 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:01.762168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 879/1743] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:01.766468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81427700-da25-45df-a6b1-a190c72c1a32 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:01.774045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-81427700-da25-45df-a6b1-a190c72c1a32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:01.795465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 865/1744] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:01 2022] PUT /identity/v3/projects/6883dcfdc0954c199771ff4aca551997/users/b21a0291c3c8483db4dd56b2547c26b1/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:01.797914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ccfad-f325-4ce1-a29b-3f550c7d9393 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:01.810666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81427700-da25-45df-a6b1-a190c72c1a32 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:02.010411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ccfad-f325-4ce1-a29b-3f550c7d9393 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:02.018190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcaedc-9075-4a1d-abc3-5c92dc339d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:02.021182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcaedc-9075-4a1d-abc3-5c92dc339d23 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:02.021684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcaedc-9075-4a1d-abc3-5c92dc339d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:02.021833 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcaedc-9075-4a1d-abc3-5c92dc339d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:02.024602 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcaedc-9075-4a1d-abc3-5c92dc339d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:02.026257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcaedc-9075-4a1d-abc3-5c92dc339d23 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:02.075778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311a72f-51c9-4c90-896e-f31da52835b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:02.076576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311a72f-51c9-4c90-896e-f31da52835b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:02.076846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311a72f-51c9-4c90-896e-f31da52835b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:02.077123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311a72f-51c9-4c90-896e-f31da52835b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:02.079496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 866/1745] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:02.084199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcaedc-9075-4a1d-abc3-5c92dc339d23 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:02.105995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311a72f-51c9-4c90-896e-f31da52835b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:02.107848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311a72f-51c9-4c90-896e-f31da52835b9 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:02.117518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcaedc-9075-4a1d-abc3-5c92dc339d23 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:02.124200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c986eee-747b-4409-aa21-8052e16b6d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:02.124976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c986eee-747b-4409-aa21-8052e16b6d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:02.125253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c986eee-747b-4409-aa21-8052e16b6d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:02.125538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c986eee-747b-4409-aa21-8052e16b6d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:02.126477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c986eee-747b-4409-aa21-8052e16b6d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:02.128708 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c986eee-747b-4409-aa21-8052e16b6d5d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:02.147861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 867/1746] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:02.148153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 880/1747] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:02.149048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c986eee-747b-4409-aa21-8052e16b6d5d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:02.157751 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311a72f-51c9-4c90-896e-f31da52835b9 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:02.168263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311a72f-51c9-4c90-896e-f31da52835b9 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:02.173186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c986eee-747b-4409-aa21-8052e16b6d5d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:02.176998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052297d3-f942-4fa1-8fd6-e01a65783eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:02.178685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052297d3-f942-4fa1-8fd6-e01a65783eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:02.179096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052297d3-f942-4fa1-8fd6-e01a65783eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:02.179506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052297d3-f942-4fa1-8fd6-e01a65783eaa admin admin] PATH_INFO: `/v3/projects/6883dcfdc0954c199771ff4aca551997/users/b21a0291c3c8483db4dd56b2547c26b1/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:02.186717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052297d3-f942-4fa1-8fd6-e01a65783eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6883dcfdc0954c199771ff4aca551997, user_id=b21a0291c3c8483db4dd56b2547c26b1, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:02.189297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18356d21-fe72-44b1-bac5-3ca8ee36225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:02.190278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052297d3-f942-4fa1-8fd6-e01a65783eaa admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:02.190399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18356d21-fe72-44b1-bac5-3ca8ee36225c admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:02.190615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18356d21-fe72-44b1-bac5-3ca8ee36225c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:02.190982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18356d21-fe72-44b1-bac5-3ca8ee36225c admin admin] PATH_INFO: `/v3/projects/66c6b679f279407ab7a6e3ddace7ce51/users/7e3ad9b60e3846adaf6a688aae2ad8a0/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:02.237997 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 881/1748] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:02 2022] PUT /identity/v3/projects/6883dcfdc0954c199771ff4aca551997/users/b21a0291c3c8483db4dd56b2547c26b1/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:02.238145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18356d21-fe72-44b1-bac5-3ca8ee36225c admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c6b679f279407ab7a6e3ddace7ce51, user_id=7e3ad9b60e3846adaf6a688aae2ad8a0, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:02.249419 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18356d21-fe72-44b1-bac5-3ca8ee36225c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:02.251563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46b99f-2988-4cce-8c5e-f33aecbfae66 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:02.251855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46b99f-2988-4cce-8c5e-f33aecbfae66 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:02.252129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46b99f-2988-4cce-8c5e-f33aecbfae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:02.267174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:02.290267 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 868/1749] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:02 2022] PUT /identity/v3/projects/66c6b679f279407ab7a6e3ddace7ce51/users/7e3ad9b60e3846adaf6a688aae2ad8a0/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:02.302749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c61c4-83f2-42b6-a7da-310a6a10a574 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:02.303045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c61c4-83f2-42b6-a7da-310a6a10a574 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:02.303605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c61c4-83f2-42b6-a7da-310a6a10a574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:02.329337 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-dc46b99f-2988-4cce-8c5e-f33aecbfae66 None None] MFA Rules not processed for user `b21a0291c3c8483db4dd56b2547c26b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:02.330615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-dc46b99f-2988-4cce-8c5e-f33aecbfae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:02.394755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e52c61c4-83f2-42b6-a7da-310a6a10a574 None None] MFA Rules not processed for user `7e3ad9b60e3846adaf6a688aae2ad8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:02.396014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e52c61c4-83f2-42b6-a7da-310a6a10a574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:02.476225 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:02.536528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 882/1750] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:02.548363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46b99f-2988-4cce-8c5e-f33aecbfae66 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:02.550261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-dc46b99f-2988-4cce-8c5e-f33aecbfae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:02.589373 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 869/1751] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:15:02.646849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46b99f-2988-4cce-8c5e-f33aecbfae66 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:02.652227 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac9eb7-fd2a-4889-9e65-91aa4349a145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:02.652404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac9eb7-fd2a-4889-9e65-91aa4349a145 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:02.652686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac9eb7-fd2a-4889-9e65-91aa4349a145 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:02.653018 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac9eb7-fd2a-4889-9e65-91aa4349a145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:02.665182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac9eb7-fd2a-4889-9e65-91aa4349a145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:02.667167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac9eb7-fd2a-4889-9e65-91aa4349a145 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:02.680095 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 883/1752] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:02 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:15:03.029238 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c61c4-83f2-42b6-a7da-310a6a10a574 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:03.031603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e52c61c4-83f2-42b6-a7da-310a6a10a574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:03.178451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac9eb7-fd2a-4889-9e65-91aa4349a145 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:03.180515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-68ac9eb7-fd2a-4889-9e65-91aa4349a145 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:03.212802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c61c4-83f2-42b6-a7da-310a6a10a574 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:03.225870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:03.226855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:03.227138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:03.227413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] PATH_INFO: `/v3/projects/d9c43aa6c3f64751a0e268bebe9625c3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:03.230959 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9c43aa6c3f64751a0e268bebe9625c3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:03.232925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:03.365465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9c43aa6c3f64751a0e268bebe9625c3', 'request_id': 'req-e8a993f6-f4a8-414e-9500-18e34d8d6964'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:03.397407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9c43aa6c3f64751a0e268bebe9625c3', 'request_id': 'req-e8a993f6-f4a8-414e-9500-18e34d8d6964'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:03.411462 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9c43aa6c3f64751a0e268bebe9625c3', 'request_id': 'req-e8a993f6-f4a8-414e-9500-18e34d8d6964'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:03.413860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac9eb7-fd2a-4889-9e65-91aa4349a145 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:03.469665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] The token cache is being invalidate because project d9c43aa6c3f64751a0e268bebe9625c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:03.470240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:03.471670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:03.475923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 870/1753] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:03 2022] DELETE /identity/v3/projects/d9c43aa6c3f64751a0e268bebe9625c3 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:03.480543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:03.485127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:03.522432 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:03.628079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a0eb6-97e1-49b6-82ed-37cd9750bc6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:03.628771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a0eb6-97e1-49b6-82ed-37cd9750bc6f service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:03.629055 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a0eb6-97e1-49b6-82ed-37cd9750bc6f service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:03.629333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a0eb6-97e1-49b6-82ed-37cd9750bc6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:03.648001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a77a0eb6-97e1-49b6-82ed-37cd9750bc6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:03.659283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:03.684559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a993f6-f4a8-414e-9500-18e34d8d6964 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:03.716071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:03.717000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:03.717288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:03.717569 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] PATH_INFO: `/v3/users/0caad1fac8b64ba0971a4a184621af42` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:03.719880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0caad1fac8b64ba0971a4a184621af42)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:03.721868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:03.809291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a0eb6-97e1-49b6-82ed-37cd9750bc6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:03.812606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a0eb6-97e1-49b6-82ed-37cd9750bc6f service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:03.827433 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 884/1754] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:15:03.892707 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0caad1fac8b64ba0971a4a184621af42'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:03.895485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0caad1fac8b64ba0971a4a184621af42'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:03.901482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0caad1fac8b64ba0971a4a184621af42'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:03.931376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0caad1fac8b64ba0971a4a184621af42'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:03.949832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0caad1fac8b64ba0971a4a184621af42'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:04.015422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 871/1755] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:03 2022] DELETE /identity/v3/users/0caad1fac8b64ba0971a4a184621af42 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:04.446956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a0eb6-97e1-49b6-82ed-37cd9750bc6f service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:04.448584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a77a0eb6-97e1-49b6-82ed-37cd9750bc6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:04.472970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a0eb6-97e1-49b6-82ed-37cd9750bc6f service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:04.670023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6f205-339a-45e0-86bb-bfdee2283f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:04.670943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6f205-339a-45e0-86bb-bfdee2283f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:04.671221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6f205-339a-45e0-86bb-bfdee2283f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:04.671689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6f205-339a-45e0-86bb-bfdee2283f7c admin admin] PATH_INFO: `/v3/users/7e3ad9b60e3846adaf6a688aae2ad8a0` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:04.683084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6f205-339a-45e0-86bb-bfdee2283f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3ad9b60e3846adaf6a688aae2ad8a0)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:04.685232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6f205-339a-45e0-86bb-bfdee2283f7c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:04.779576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d0c6f205-339a-45e0-86bb-bfdee2283f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3ad9b60e3846adaf6a688aae2ad8a0'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:04.781877 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d0c6f205-339a-45e0-86bb-bfdee2283f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3ad9b60e3846adaf6a688aae2ad8a0'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:04.803916 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d0c6f205-339a-45e0-86bb-bfdee2283f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3ad9b60e3846adaf6a688aae2ad8a0'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:04.820756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d0c6f205-339a-45e0-86bb-bfdee2283f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3ad9b60e3846adaf6a688aae2ad8a0'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:04.824991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d0c6f205-339a-45e0-86bb-bfdee2283f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3ad9b60e3846adaf6a688aae2ad8a0'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:04.841294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:04.864076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:04.866157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:04.887371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586d849-201c-4ed2-a0e2-b46a15d42716 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:04.906091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a2484-3d6e-4743-bad7-1c90fa3c5a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:04.913089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a2484-3d6e-4743-bad7-1c90fa3c5a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:04.913388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a2484-3d6e-4743-bad7-1c90fa3c5a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:04.916051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a2484-3d6e-4743-bad7-1c90fa3c5a3f admin admin] PATH_INFO: `/v3/users/0208b6c80c0f4e4e9e093286b0843732` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:04.921038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a2484-3d6e-4743-bad7-1c90fa3c5a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0208b6c80c0f4e4e9e093286b0843732)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:04.923337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a2484-3d6e-4743-bad7-1c90fa3c5a3f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:04.927304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 885/1756] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:04 2022] DELETE /identity/v3/users/7e3ad9b60e3846adaf6a688aae2ad8a0 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:04.932037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342ba51-d438-4e5c-83e5-d293cf09d1a6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:04.938233 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342ba51-d438-4e5c-83e5-d293cf09d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:04.938726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342ba51-d438-4e5c-83e5-d293cf09d1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:05.020234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5342ba51-d438-4e5c-83e5-d293cf09d1a6 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:05.033784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5342ba51-d438-4e5c-83e5-d293cf09d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:05.069801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8f9a2484-3d6e-4743-bad7-1c90fa3c5a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0208b6c80c0f4e4e9e093286b0843732'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:05.071404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8f9a2484-3d6e-4743-bad7-1c90fa3c5a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0208b6c80c0f4e4e9e093286b0843732'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:05.073374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8f9a2484-3d6e-4743-bad7-1c90fa3c5a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0208b6c80c0f4e4e9e093286b0843732'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:05.095976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8f9a2484-3d6e-4743-bad7-1c90fa3c5a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0208b6c80c0f4e4e9e093286b0843732'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:05.110503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8f9a2484-3d6e-4743-bad7-1c90fa3c5a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0208b6c80c0f4e4e9e093286b0843732'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:05.160898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 872/1757] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:04 2022] DELETE /identity/v3/users/0208b6c80c0f4e4e9e093286b0843732 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:05.173817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876edc-d8e4-45ae-b297-3ac5dd1c69ac None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:05.174139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876edc-d8e4-45ae-b297-3ac5dd1c69ac None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:05.174450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876edc-d8e4-45ae-b297-3ac5dd1c69ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:05.215672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-08876edc-d8e4-45ae-b297-3ac5dd1c69ac None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:05.218502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-08876edc-d8e4-45ae-b297-3ac5dd1c69ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:05.275237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 886/1758] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:05.276208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342ba51-d438-4e5c-83e5-d293cf09d1a6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:05.281107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5342ba51-d438-4e5c-83e5-d293cf09d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:05.305855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342ba51-d438-4e5c-83e5-d293cf09d1a6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:05.344968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:05.410294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 873/1759] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:05.412101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876edc-d8e4-45ae-b297-3ac5dd1c69ac None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:05.417770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-08876edc-d8e4-45ae-b297-3ac5dd1c69ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:05.421932 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:05.423859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:05.426201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:05.426566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] PATH_INFO: `/v3/projects/5daa0d76de7c46d49d865277904e98fa` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:05.430836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5daa0d76de7c46d49d865277904e98fa)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:05.434910 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:05.591665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876edc-d8e4-45ae-b297-3ac5dd1c69ac None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:05.604406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5daa0d76de7c46d49d865277904e98fa', 'request_id': 'req-e28bf389-feac-417f-9d9f-16d67f816174'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:05.621468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5daa0d76de7c46d49d865277904e98fa', 'request_id': 'req-e28bf389-feac-417f-9d9f-16d67f816174'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:05.639446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5daa0d76de7c46d49d865277904e98fa', 'request_id': 'req-e28bf389-feac-417f-9d9f-16d67f816174'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:05.662087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] The token cache is being invalidate because project 5daa0d76de7c46d49d865277904e98fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:05.662383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:05.663045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:05.664656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 887/1760] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:05 2022] DELETE /identity/v3/projects/5daa0d76de7c46d49d865277904e98fa => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:05.668433 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:05.671113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:05.691529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428686-ab29-4a77-8d34-1be873ddee78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:05.692210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428686-ab29-4a77-8d34-1be873ddee78 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:05.692493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428686-ab29-4a77-8d34-1be873ddee78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:05.692766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428686-ab29-4a77-8d34-1be873ddee78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:05.694728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6c428686-ab29-4a77-8d34-1be873ddee78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:05.708591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:05.760931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bf389-feac-417f-9d9f-16d67f816174 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:05.766477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33075cc4-b1b5-4bfa-8d06-01dcde8e8392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:05.767450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33075cc4-b1b5-4bfa-8d06-01dcde8e8392 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:05.767747 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33075cc4-b1b5-4bfa-8d06-01dcde8e8392 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:05.768059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33075cc4-b1b5-4bfa-8d06-01dcde8e8392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:05.773491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-33075cc4-b1b5-4bfa-8d06-01dcde8e8392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:05.791778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c428686-ab29-4a77-8d34-1be873ddee78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:05.793935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c428686-ab29-4a77-8d34-1be873ddee78 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:05.809983 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 874/1761] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:05.811260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428686-ab29-4a77-8d34-1be873ddee78 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:05.821780 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428686-ab29-4a77-8d34-1be873ddee78 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:05.826994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a35b0d-7c13-4d99-8ad0-eb1b703c1ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:05.827604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a35b0d-7c13-4d99-8ad0-eb1b703c1ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:05.827970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a35b0d-7c13-4d99-8ad0-eb1b703c1ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:05.828236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a35b0d-7c13-4d99-8ad0-eb1b703c1ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:05.852653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-42a35b0d-7c13-4d99-8ad0-eb1b703c1ef2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:05.913512 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33075cc4-b1b5-4bfa-8d06-01dcde8e8392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:05.916717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33075cc4-b1b5-4bfa-8d06-01dcde8e8392 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:05.939684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 888/1762] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:05.941100 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6764c-3691-4e8f-8c21-fbd50e9a646d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:05.941211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6764c-3691-4e8f-8c21-fbd50e9a646d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:05.941442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6764c-3691-4e8f-8c21-fbd50e9a646d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:05.970006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:05.989326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-86a6764c-3691-4e8f-8c21-fbd50e9a646d None None] MFA Rules not processed for user `43bcd8fb25604011bfd1df823ea4ddad`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:05.991989 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-86a6764c-3691-4e8f-8c21-fbd50e9a646d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:06.000445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a35b0d-7c13-4d99-8ad0-eb1b703c1ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:06.001941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a35b0d-7c13-4d99-8ad0-eb1b703c1ef2 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:06.038370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 875/1763] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:06.050542 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e7f74-8b50-4084-bee1-50ff1b3f2ebd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:06.050838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e7f74-8b50-4084-bee1-50ff1b3f2ebd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:06.051124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e7f74-8b50-4084-bee1-50ff1b3f2ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:06.106990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-399e7f74-8b50-4084-bee1-50ff1b3f2ebd None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:06.108270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-399e7f74-8b50-4084-bee1-50ff1b3f2ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:06.165869 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 876/1764] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:06.167485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d347618-9119-4bb7-b83d-502191cb846d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:06.167855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d347618-9119-4bb7-b83d-502191cb846d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:06.168181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d347618-9119-4bb7-b83d-502191cb846d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:06.187576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 889/1765] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:06.196861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6d347618-9119-4bb7-b83d-502191cb846d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:06.198854 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6d347618-9119-4bb7-b83d-502191cb846d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:06.209072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 877/1766] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:06.213677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6764c-3691-4e8f-8c21-fbd50e9a646d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:06.236890 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6764c-3691-4e8f-8c21-fbd50e9a646d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:06.261538 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c119115-a6fb-4950-b58b-9b374eda367c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:06.262389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c119115-a6fb-4950-b58b-9b374eda367c admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:06.262681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c119115-a6fb-4950-b58b-9b374eda367c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:06.262963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c119115-a6fb-4950-b58b-9b374eda367c admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:06.263713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c119115-a6fb-4950-b58b-9b374eda367c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:06.264024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c119115-a6fb-4950-b58b-9b374eda367c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:06.274583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c119115-a6fb-4950-b58b-9b374eda367c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:06.340583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 890/1767] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:06.342415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d347618-9119-4bb7-b83d-502191cb846d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:06.366892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d347618-9119-4bb7-b83d-502191cb846d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:06.372803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fbff1-5d6c-4377-b58e-256f8edf8cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:06.373684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fbff1-5d6c-4377-b58e-256f8edf8cfb admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:06.374081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fbff1-5d6c-4377-b58e-256f8edf8cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:06.374440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fbff1-5d6c-4377-b58e-256f8edf8cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:06.375393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fbff1-5d6c-4377-b58e-256f8edf8cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:06.395934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fbff1-5d6c-4377-b58e-256f8edf8cfb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:06.469095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 878/1768] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:06 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:06.473289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c119115-a6fb-4950-b58b-9b374eda367c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:06.502605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c119115-a6fb-4950-b58b-9b374eda367c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:06.675316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30726b0c-558d-4600-854c-f8a86c6bb143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:06.677428 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30726b0c-558d-4600-854c-f8a86c6bb143 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:06.678595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30726b0c-558d-4600-854c-f8a86c6bb143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:06.679582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30726b0c-558d-4600-854c-f8a86c6bb143 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:06.681480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30726b0c-558d-4600-854c-f8a86c6bb143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:06.689064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30726b0c-558d-4600-854c-f8a86c6bb143 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:06.714316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:06.752759 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 891/1769] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:06 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:06.757002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fbff1-5d6c-4377-b58e-256f8edf8cfb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:06.772583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fbff1-5d6c-4377-b58e-256f8edf8cfb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:06.793152 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda54b01-d65a-480e-acc7-c9de8b2d40d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:06.793903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda54b01-d65a-480e-acc7-c9de8b2d40d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:06.794199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda54b01-d65a-480e-acc7-c9de8b2d40d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:06.794486 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda54b01-d65a-480e-acc7-c9de8b2d40d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:06.795312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda54b01-d65a-480e-acc7-c9de8b2d40d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:06.797199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda54b01-d65a-480e-acc7-c9de8b2d40d2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:06.821214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 879/1770] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:06.827020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30726b0c-558d-4600-854c-f8a86c6bb143 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:06.842401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30726b0c-558d-4600-854c-f8a86c6bb143 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:06.867514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24e2e2-9fdf-44a2-bfbd-d8a84b52b768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:06.868145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24e2e2-9fdf-44a2-bfbd-d8a84b52b768 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:06.868427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24e2e2-9fdf-44a2-bfbd-d8a84b52b768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:06.868712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24e2e2-9fdf-44a2-bfbd-d8a84b52b768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:06.871123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24e2e2-9fdf-44a2-bfbd-d8a84b52b768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:06.875327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24e2e2-9fdf-44a2-bfbd-d8a84b52b768 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:06.909591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 892/1771] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:06.913066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda54b01-d65a-480e-acc7-c9de8b2d40d2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:06.941151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda54b01-d65a-480e-acc7-c9de8b2d40d2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:06.959700 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8de8ef-e28c-45e2-9574-dfa537f7db4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:06.963071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de8ef-e28c-45e2-9574-dfa537f7db4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:06.965479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de8ef-e28c-45e2-9574-dfa537f7db4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:06.965811 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8de8ef-e28c-45e2-9574-dfa537f7db4b admin admin] PATH_INFO: `/v3/projects/4b9804d46bef4df5a33201156023dbb3/users/c809c1a6bf3f41948a0b9171f30ac1e5/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:07.070078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8de8ef-e28c-45e2-9574-dfa537f7db4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9804d46bef4df5a33201156023dbb3, user_id=c809c1a6bf3f41948a0b9171f30ac1e5, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:07.075339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8de8ef-e28c-45e2-9574-dfa537f7db4b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:07.097430 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 880/1772] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:06 2022] PUT /identity/v3/projects/4b9804d46bef4df5a33201156023dbb3/users/c809c1a6bf3f41948a0b9171f30ac1e5/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:07.107686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296e3e-c7aa-4cb7-baf2-4eb85dbc159a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:07.108271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296e3e-c7aa-4cb7-baf2-4eb85dbc159a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:07.108552 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296e3e-c7aa-4cb7-baf2-4eb85dbc159a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:07.142145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2e296e3e-c7aa-4cb7-baf2-4eb85dbc159a None None] MFA Rules not processed for user `c809c1a6bf3f41948a0b9171f30ac1e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:07.143951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2e296e3e-c7aa-4cb7-baf2-4eb85dbc159a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:07.237949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 893/1773] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:07.240862 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8de8ef-e28c-45e2-9574-dfa537f7db4b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:07.363986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8de8ef-e28c-45e2-9574-dfa537f7db4b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:07.370001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacaac1-3184-4392-81f2-35b08a35c600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:07.370868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacaac1-3184-4392-81f2-35b08a35c600 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:07.371257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacaac1-3184-4392-81f2-35b08a35c600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:07.371772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacaac1-3184-4392-81f2-35b08a35c600 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:07.372741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddacaac1-3184-4392-81f2-35b08a35c600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:07.376022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddacaac1-3184-4392-81f2-35b08a35c600 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:07.420575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 881/1774] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:07 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:07.421338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296e3e-c7aa-4cb7-baf2-4eb85dbc159a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:07.436657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296e3e-c7aa-4cb7-baf2-4eb85dbc159a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:07.523360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a0042-d1b2-4325-877d-2f0976aea8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:07.524139 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a0042-d1b2-4325-877d-2f0976aea8c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:07.524493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a0042-d1b2-4325-877d-2f0976aea8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:07.524881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a0042-d1b2-4325-877d-2f0976aea8c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:07.525670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a0042-d1b2-4325-877d-2f0976aea8c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:07.530256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a0042-d1b2-4325-877d-2f0976aea8c6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:07.561576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 894/1775] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:07 2022] POST /identity/v3/users => generated 294 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:07.566299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacaac1-3184-4392-81f2-35b08a35c600 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:07.578691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a0042-d1b2-4325-877d-2f0976aea8c6 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:07.582221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1b1a0042-d1b2-4325-877d-2f0976aea8c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:07.582359 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacaac1-3184-4392-81f2-35b08a35c600 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:07.595671 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331bf81-eb08-4241-bb00-25d868db1a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:07.595671 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331bf81-eb08-4241-bb00-25d868db1a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:07.598299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331bf81-eb08-4241-bb00-25d868db1a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:07.598633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331bf81-eb08-4241-bb00-25d868db1a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:07.599563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a0042-d1b2-4325-877d-2f0976aea8c6 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:07.599762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331bf81-eb08-4241-bb00-25d868db1a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:07.604219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331bf81-eb08-4241-bb00-25d868db1a3a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:07.615228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:07.618109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:07.618289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:07.618583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] PATH_INFO: `/v3/projects/66c6b679f279407ab7a6e3ddace7ce51` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:07.620164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c6b679f279407ab7a6e3ddace7ce51)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:07.623346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:07.664753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 882/1776] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:07.668600 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331bf81-eb08-4241-bb00-25d868db1a3a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:07.691734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331bf81-eb08-4241-bb00-25d868db1a3a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:07.698351 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69432dfc-b47e-474f-9462-29ce7f8af3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:07.699293 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69432dfc-b47e-474f-9462-29ce7f8af3eb admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:07.699663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69432dfc-b47e-474f-9462-29ce7f8af3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:07.700835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69432dfc-b47e-474f-9462-29ce7f8af3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:07.701816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69432dfc-b47e-474f-9462-29ce7f8af3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:07.707364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69432dfc-b47e-474f-9462-29ce7f8af3eb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:07.737135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 883/1777] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:07.739525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69432dfc-b47e-474f-9462-29ce7f8af3eb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:07.740241 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c6b679f279407ab7a6e3ddace7ce51', 'request_id': 'req-2338bdf1-96a1-48eb-9d23-d9f842042012'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:07.757520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:INSERT=7,keystone:SELECT=89,keystone:DELETE=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:07.773073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c6b679f279407ab7a6e3ddace7ce51', 'request_id': 'req-2338bdf1-96a1-48eb-9d23-d9f842042012'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:07.777681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69432dfc-b47e-474f-9462-29ce7f8af3eb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:07.786773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c6b679f279407ab7a6e3ddace7ce51', 'request_id': 'req-2338bdf1-96a1-48eb-9d23-d9f842042012'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:07.823681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] The token cache is being invalidate because project 66c6b679f279407ab7a6e3ddace7ce51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:07.823979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:07.829187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:07.831392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 895/1778] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:07 2022] DELETE /identity/v3/projects/66c6b679f279407ab7a6e3ddace7ce51 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:07.834601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:07.835445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:07.903763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:07.946569 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220da12f-61e0-48bb-90d6-231c36b3a046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:07.948635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220da12f-61e0-48bb-90d6-231c36b3a046 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:07.950394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220da12f-61e0-48bb-90d6-231c36b3a046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:07.950704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220da12f-61e0-48bb-90d6-231c36b3a046 admin admin] PATH_INFO: `/v3/projects/107bbbb5785047dc85b7aa5bdf532c85/users/38c0322c2d0c4d5db70d19fd159de037/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:07.972159 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220da12f-61e0-48bb-90d6-231c36b3a046 admin admin] RBAC: Authorizing `identity:create_grant(project_id=107bbbb5785047dc85b7aa5bdf532c85, user_id=38c0322c2d0c4d5db70d19fd159de037, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:07.975287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220da12f-61e0-48bb-90d6-231c36b3a046 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:07.986455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338bdf1-96a1-48eb-9d23-d9f842042012 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:08.004315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 884/1779] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:07 2022] PUT /identity/v3/projects/107bbbb5785047dc85b7aa5bdf532c85/users/38c0322c2d0c4d5db70d19fd159de037/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:08.008048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29026a-dbec-4475-9225-a868dca52473 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:08.008893 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29026a-dbec-4475-9225-a868dca52473 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:08.010372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29026a-dbec-4475-9225-a868dca52473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:08.068287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8c29026a-dbec-4475-9225-a868dca52473 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:08.069496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8c29026a-dbec-4475-9225-a868dca52473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:08.167166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:08.168182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:08.168476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:08.168835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] PATH_INFO: `/v3/projects/00794e1916234e4b8e20037eec608eb0` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:08.171852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00794e1916234e4b8e20037eec608eb0)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:08.179702 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:08.213862 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 885/1780] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:08.216601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e13596-5ea2-4775-b7b3-61bff0b026a6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:08.217000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e13596-5ea2-4775-b7b3-61bff0b026a6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:08.217258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e13596-5ea2-4775-b7b3-61bff0b026a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:08.257843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b8e13596-5ea2-4775-b7b3-61bff0b026a6 None None] MFA Rules not processed for user `38c0322c2d0c4d5db70d19fd159de037`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:08.259461 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b8e13596-5ea2-4775-b7b3-61bff0b026a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:08.340125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00794e1916234e4b8e20037eec608eb0', 'request_id': 'req-2b3e4cca-76df-420a-ab55-78a3e7e07a73'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.351306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00794e1916234e4b8e20037eec608eb0', 'request_id': 'req-2b3e4cca-76df-420a-ab55-78a3e7e07a73'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.363300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00794e1916234e4b8e20037eec608eb0', 'request_id': 'req-2b3e4cca-76df-420a-ab55-78a3e7e07a73'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.377556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] The token cache is being invalidate because project 00794e1916234e4b8e20037eec608eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:08.377875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.379469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2b3e4cca-76df-420a-ab55-78a3e7e07a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.382629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 896/1781] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:07 2022] DELETE /identity/v3/projects/00794e1916234e4b8e20037eec608eb0 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:08.385166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884831c-9467-4987-a7d9-7bcf7c59c74f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:08.385438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884831c-9467-4987-a7d9-7bcf7c59c74f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:08.385769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884831c-9467-4987-a7d9-7bcf7c59c74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:08.408716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 886/1782] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:08.409484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e13596-5ea2-4775-b7b3-61bff0b026a6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:08.419344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b8e13596-5ea2-4775-b7b3-61bff0b026a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:08.436772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9884831c-9467-4987-a7d9-7bcf7c59c74f None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:08.438068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9884831c-9467-4987-a7d9-7bcf7c59c74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:08.453556 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e13596-5ea2-4775-b7b3-61bff0b026a6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:08.467586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:08.574549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:08.588694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 897/1783] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:08.590034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884831c-9467-4987-a7d9-7bcf7c59c74f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:08.591164 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:08.592233 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:08.592525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:08.592817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] PATH_INFO: `/v3/users/13ae9af323874e0eb4807f02c9d6ed31` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:08.593867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9884831c-9467-4987-a7d9-7bcf7c59c74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:08.594105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ae9af323874e0eb4807f02c9d6ed31)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:08.596236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:08.607568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884831c-9467-4987-a7d9-7bcf7c59c74f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:08.612991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:08.614221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:08.614611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:08.615011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] PATH_INFO: `/v3/users/b21a0291c3c8483db4dd56b2547c26b1` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:08.616385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b21a0291c3c8483db4dd56b2547c26b1)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:08.618517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:08.677278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ae9af323874e0eb4807f02c9d6ed31'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.678938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ae9af323874e0eb4807f02c9d6ed31'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.679959 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ae9af323874e0eb4807f02c9d6ed31'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.691750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ae9af323874e0eb4807f02c9d6ed31'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.706499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ae9af323874e0eb4807f02c9d6ed31'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.734233 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b21a0291c3c8483db4dd56b2547c26b1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.734948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b21a0291c3c8483db4dd56b2547c26b1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.746902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 887/1784] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:08 2022] DELETE /identity/v3/users/13ae9af323874e0eb4807f02c9d6ed31 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:08.747694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:08.749101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:08.750868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b21a0291c3c8483db4dd56b2547c26b1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.765616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b21a0291c3c8483db4dd56b2547c26b1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.766672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b21a0291c3c8483db4dd56b2547c26b1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:08.796053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 898/1785] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:08 2022] DELETE /identity/v3/users/b21a0291c3c8483db4dd56b2547c26b1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:08.797741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:08.804305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:08.872719 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676f637-e108-4215-91fa-ae45024000ee admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:08.916044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea543a0d-3dc3-4dfd-a081-57972cdadc75 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:08.927190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f103f-9d68-4956-80df-d822c2c39676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:08.927878 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f103f-9d68-4956-80df-d822c2c39676 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:08.928159 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f103f-9d68-4956-80df-d822c2c39676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:08.928438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f103f-9d68-4956-80df-d822c2c39676 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:08.929165 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f103f-9d68-4956-80df-d822c2c39676 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:08.929468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f103f-9d68-4956-80df-d822c2c39676 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:08.931132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f103f-9d68-4956-80df-d822c2c39676 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:08.948055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6adbc-91b2-46d4-9386-a73b9d54d30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:08.948055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6adbc-91b2-46d4-9386-a73b9d54d30e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:08.948055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6adbc-91b2-46d4-9386-a73b9d54d30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:08.948581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6adbc-91b2-46d4-9386-a73b9d54d30e admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:08.949377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6adbc-91b2-46d4-9386-a73b9d54d30e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:08.949967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 899/1786] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:08.951448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6adbc-91b2-46d4-9386-a73b9d54d30e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:08.953097 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70978ae1-8760-485f-9873-4a9ca81296f7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:08.953515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70978ae1-8760-485f-9873-4a9ca81296f7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:08.953697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70978ae1-8760-485f-9873-4a9ca81296f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:08.974539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-70978ae1-8760-485f-9873-4a9ca81296f7 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:08.975738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-70978ae1-8760-485f-9873-4a9ca81296f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:08.986183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 888/1787] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:08 2022] POST /identity/v3/projects => generated 387 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:08.986183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6adbc-91b2-46d4-9386-a73b9d54d30e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:08.991571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:08.998998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 900/1788] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:09.003049 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70978ae1-8760-485f-9873-4a9ca81296f7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.004495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-70978ae1-8760-485f-9873-4a9ca81296f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:09.018521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70978ae1-8760-485f-9873-4a9ca81296f7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.033700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:09.069999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6adbc-91b2-46d4-9386-a73b9d54d30e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.075223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0709117-1950-4141-81b4-5473da69dcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.075913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0709117-1950-4141-81b4-5473da69dcfd admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.076543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0709117-1950-4141-81b4-5473da69dcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.076543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0709117-1950-4141-81b4-5473da69dcfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.077153 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0709117-1950-4141-81b4-5473da69dcfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.079221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0709117-1950-4141-81b4-5473da69dcfd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.105721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9b046-8e8d-47d2-9f67-3c5bcd03abe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.106580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b046-8e8d-47d2-9f67-3c5bcd03abe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.106749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b046-8e8d-47d2-9f67-3c5bcd03abe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.107362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9b046-8e8d-47d2-9f67-3c5bcd03abe8 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.108162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9b046-8e8d-47d2-9f67-3c5bcd03abe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.110343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9b046-8e8d-47d2-9f67-3c5bcd03abe8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.110471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 889/1789] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:08 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:09.114598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0709117-1950-4141-81b4-5473da69dcfd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.116190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a0709117-1950-4141-81b4-5473da69dcfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:09.140398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 901/1790] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:08 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:09.142386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9b046-8e8d-47d2-9f67-3c5bcd03abe8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.161639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9b046-8e8d-47d2-9f67-3c5bcd03abe8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.188577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0709117-1950-4141-81b4-5473da69dcfd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.192934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b3709-f07a-4669-a02b-521ed1a4a42c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.193710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b3709-f07a-4669-a02b-521ed1a4a42c service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.194104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b3709-f07a-4669-a02b-521ed1a4a42c service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.194465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b3709-f07a-4669-a02b-521ed1a4a42c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.261041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb060bc0-931e-45b0-b38c-13f9d672eb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.262046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb060bc0-931e-45b0-b38c-13f9d672eb20 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.262405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb060bc0-931e-45b0-b38c-13f9d672eb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.262748 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb060bc0-931e-45b0-b38c-13f9d672eb20 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.263535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb060bc0-931e-45b0-b38c-13f9d672eb20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.265561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb060bc0-931e-45b0-b38c-13f9d672eb20 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.280064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b3709-f07a-4669-a02b-521ed1a4a42c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.281840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b3709-f07a-4669-a02b-521ed1a4a42c service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.286923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 902/1791] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:09 2022] POST /identity/v3/users => generated 283 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:09.287770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb060bc0-931e-45b0-b38c-13f9d672eb20 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.292899 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 890/1792] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:09.293779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b3709-f07a-4669-a02b-521ed1a4a42c service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.296173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0d7b3709-f07a-4669-a02b-521ed1a4a42c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:09.299448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb060bc0-931e-45b0-b38c-13f9d672eb20 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.304660 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8737b-dbd9-487b-a692-4e0de1f760d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.305391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8737b-dbd9-487b-a692-4e0de1f760d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.305672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8737b-dbd9-487b-a692-4e0de1f760d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.305983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8737b-dbd9-487b-a692-4e0de1f760d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.306720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8737b-dbd9-487b-a692-4e0de1f760d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.308995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8737b-dbd9-487b-a692-4e0de1f760d5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.309323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b3709-f07a-4669-a02b-521ed1a4a42c service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.315039 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.315969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.316257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.316560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] PATH_INFO: `/v3/projects/6883dcfdc0954c199771ff4aca551997` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.317694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6883dcfdc0954c199771ff4aca551997)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.319866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.320295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 903/1793] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:09.320947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8737b-dbd9-487b-a692-4e0de1f760d5 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.331950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8737b-dbd9-487b-a692-4e0de1f760d5 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.336817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca59a2f-b6a4-4d9f-92ff-2a5a3021944a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.337540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca59a2f-b6a4-4d9f-92ff-2a5a3021944a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.337826 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca59a2f-b6a4-4d9f-92ff-2a5a3021944a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.338135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca59a2f-b6a4-4d9f-92ff-2a5a3021944a admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.338894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca59a2f-b6a4-4d9f-92ff-2a5a3021944a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.341070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca59a2f-b6a4-4d9f-92ff-2a5a3021944a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.354223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 904/1794] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:09.355306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca59a2f-b6a4-4d9f-92ff-2a5a3021944a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.367656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca59a2f-b6a4-4d9f-92ff-2a5a3021944a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.374563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:09.391104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6883dcfdc0954c199771ff4aca551997', 'request_id': 'req-3331ef02-b099-4a10-9570-1b4f0cb28355'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:09.402734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6883dcfdc0954c199771ff4aca551997', 'request_id': 'req-3331ef02-b099-4a10-9570-1b4f0cb28355'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:09.410306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6883dcfdc0954c199771ff4aca551997', 'request_id': 'req-3331ef02-b099-4a10-9570-1b4f0cb28355'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:09.422057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] The token cache is being invalidate because project 6883dcfdc0954c199771ff4aca551997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:09.422366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:09.423141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:09.425186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 891/1795] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:09 2022] DELETE /identity/v3/projects/6883dcfdc0954c199771ff4aca551997 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:09.426004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.427544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:09.436907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c6011-2a37-46c6-8314-71567445721e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.438276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c6011-2a37-46c6-8314-71567445721e admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.438276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c6011-2a37-46c6-8314-71567445721e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.438495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c6011-2a37-46c6-8314-71567445721e admin admin] PATH_INFO: `/v3/projects/81bc8df29a9e471dae661f680cc54b6a/users/0a9f35fa120e4de69364c76fab8d7827/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.457629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c6011-2a37-46c6-8314-71567445721e admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bc8df29a9e471dae661f680cc54b6a, user_id=0a9f35fa120e4de69364c76fab8d7827, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.460375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c6011-2a37-46c6-8314-71567445721e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.470147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:09.479011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 905/1796] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:09 2022] PUT /identity/v3/projects/81bc8df29a9e471dae661f680cc54b6a/users/0a9f35fa120e4de69364c76fab8d7827/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:09.480802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda886-3ca7-4a8d-ad79-9a5b65915838 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.481158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda886-3ca7-4a8d-ad79-9a5b65915838 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.481863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda886-3ca7-4a8d-ad79-9a5b65915838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.501453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-58fda886-3ca7-4a8d-ad79-9a5b65915838 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:09.502788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-58fda886-3ca7-4a8d-ad79-9a5b65915838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:09.516971 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ef02-b099-4a10-9570-1b4f0cb28355 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.568354 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 906/1797] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:09.569219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fda886-3ca7-4a8d-ad79-9a5b65915838 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.571161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-58fda886-3ca7-4a8d-ad79-9a5b65915838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:09.585726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fda886-3ca7-4a8d-ad79-9a5b65915838 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.589110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808bb99-95ea-4211-903c-6d12a1da7d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.590393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808bb99-95ea-4211-903c-6d12a1da7d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.590690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808bb99-95ea-4211-903c-6d12a1da7d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.590974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808bb99-95ea-4211-903c-6d12a1da7d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.591521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8bf96-77d8-44e5-bf43-f91e5eb0fd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.591736 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808bb99-95ea-4211-903c-6d12a1da7d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.592381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8bf96-77d8-44e5-bf43-f91e5eb0fd6b admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.592797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8bf96-77d8-44e5-bf43-f91e5eb0fd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.593223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8bf96-77d8-44e5-bf43-f91e5eb0fd6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.593362 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808bb99-95ea-4211-903c-6d12a1da7d86 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.595038 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8bf96-77d8-44e5-bf43-f91e5eb0fd6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.596857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8bf96-77d8-44e5-bf43-f91e5eb0fd6b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.625231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 892/1799] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:09.630637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f09be-7cfc-4217-a7b4-72b7d25f0386 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.630969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f09be-7cfc-4217-a7b4-72b7d25f0386 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.631958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f09be-7cfc-4217-a7b4-72b7d25f0386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.632626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 907/1799] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:09.636113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8bf96-77d8-44e5-bf43-f91e5eb0fd6b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.644948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6dc8bf96-77d8-44e5-bf43-f91e5eb0fd6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:09.655697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-001f09be-7cfc-4217-a7b4-72b7d25f0386 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:09.658279 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-001f09be-7cfc-4217-a7b4-72b7d25f0386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:09.673171 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 893/1800] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:09.674214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f09be-7cfc-4217-a7b4-72b7d25f0386 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.689194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8bf96-77d8-44e5-bf43-f91e5eb0fd6b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.690473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f09be-7cfc-4217-a7b4-72b7d25f0386 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:09.697812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e7a6-8717-4222-9519-0fafc69dc931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.699032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e7a6-8717-4222-9519-0fafc69dc931 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.699583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e7a6-8717-4222-9519-0fafc69dc931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.700836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e7a6-8717-4222-9519-0fafc69dc931 admin admin] PATH_INFO: `/v3/projects/81bc8df29a9e471dae661f680cc54b6a/users/0a9f35fa120e4de69364c76fab8d7827/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.702684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:09.703189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6e7a6-8717-4222-9519-0fafc69dc931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bc8df29a9e471dae661f680cc54b6a, user_id=0a9f35fa120e4de69364c76fab8d7827, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.703620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:09.703902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:09.705005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] PATH_INFO: `/v3/projects/f04d65b2185b4f10a065ae5bf24245ec` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:09.707236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6e7a6-8717-4222-9519-0fafc69dc931 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.710929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f04d65b2185b4f10a065ae5bf24245ec)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:09.712907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:09.721873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 894/1801] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:09 2022] PUT /identity/v3/projects/81bc8df29a9e471dae661f680cc54b6a/users/0a9f35fa120e4de69364c76fab8d7827/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:09.748850 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e7a6-8717-4222-9519-0fafc69dc931 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.862446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f04d65b2185b4f10a065ae5bf24245ec', 'request_id': 'req-baf7add2-567a-481f-b38e-29647ff6ead7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:09.882623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f04d65b2185b4f10a065ae5bf24245ec', 'request_id': 'req-baf7add2-567a-481f-b38e-29647ff6ead7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:09.900865 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f04d65b2185b4f10a065ae5bf24245ec', 'request_id': 'req-baf7add2-567a-481f-b38e-29647ff6ead7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:09.916139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:09.930150 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] The token cache is being invalidate because project f04d65b2185b4f10a065ae5bf24245ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:09.930511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:09.932855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:09.939959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 908/1802] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:09 2022] DELETE /identity/v3/projects/f04d65b2185b4f10a065ae5bf24245ec => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:09.939959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:09.943644 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:09.949729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e7a6-8717-4222-9519-0fafc69dc931 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:10.103611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7add2-567a-481f-b38e-29647ff6ead7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:10.109903 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44448a-3fb4-4ada-a6bb-3c513316fc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:10.110239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44448a-3fb4-4ada-a6bb-3c513316fc10 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:10.110533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44448a-3fb4-4ada-a6bb-3c513316fc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:10.110826 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44448a-3fb4-4ada-a6bb-3c513316fc10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:10.111572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44448a-3fb4-4ada-a6bb-3c513316fc10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:10.111893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44448a-3fb4-4ada-a6bb-3c513316fc10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:10.113503 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44448a-3fb4-4ada-a6bb-3c513316fc10 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:10.118914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e771ab-96f7-4455-ade7-f0c83d57e2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:10.119848 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e771ab-96f7-4455-ade7-f0c83d57e2d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:10.120246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e771ab-96f7-4455-ade7-f0c83d57e2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:10.120763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e771ab-96f7-4455-ade7-f0c83d57e2d0 admin admin] PATH_INFO: `/v3/projects/00fbe799557e40c0a8253f134abbdb74/users/c5c569cac8244c0ba2fe6dc3c9507e53/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:10.186822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 909/1803] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:10.188370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc108d-ad23-4b0f-84d8-fc4edfc20fc5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:10.188666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc108d-ad23-4b0f-84d8-fc4edfc20fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:10.188976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc108d-ad23-4b0f-84d8-fc4edfc20fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:10.200379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:10.211513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e771ab-96f7-4455-ade7-f0c83d57e2d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00fbe799557e40c0a8253f134abbdb74, user_id=c5c569cac8244c0ba2fe6dc3c9507e53, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:10.214536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e771ab-96f7-4455-ade7-f0c83d57e2d0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:10.230607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c9dc108d-ad23-4b0f-84d8-fc4edfc20fc5 None None] MFA Rules not processed for user `0a9f35fa120e4de69364c76fab8d7827`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:10.232018 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c9dc108d-ad23-4b0f-84d8-fc4edfc20fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:10.238329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 895/1804] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:09 2022] PUT /identity/v3/projects/00fbe799557e40c0a8253f134abbdb74/users/c5c569cac8244c0ba2fe6dc3c9507e53/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:10.240336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5f0c2-984d-4e0d-9b38-aa4a2e72b355 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:10.241208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5f0c2-984d-4e0d-9b38-aa4a2e72b355 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:10.241495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5f0c2-984d-4e0d-9b38-aa4a2e72b355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:10.268190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3cd5f0c2-984d-4e0d-9b38-aa4a2e72b355 None None] MFA Rules not processed for user `98649aa38f764af0b41ff5594b94a989`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:10.269928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3cd5f0c2-984d-4e0d-9b38-aa4a2e72b355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:10.355335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 896/1805] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Jul 4 09:15:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:10.358565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6d639-041e-4f98-9aaf-9062a9094fbb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:10.359256 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6d639-041e-4f98-9aaf-9062a9094fbb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:10.359732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6d639-041e-4f98-9aaf-9062a9094fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:10.382630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-29d6d639-041e-4f98-9aaf-9062a9094fbb None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:10.384380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-29d6d639-041e-4f98-9aaf-9062a9094fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:10.390738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 910/1806] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:10.392115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc108d-ad23-4b0f-84d8-fc4edfc20fc5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:10.394564 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c9dc108d-ad23-4b0f-84d8-fc4edfc20fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:10.486199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 897/1807] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:15:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:10.487805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6d639-041e-4f98-9aaf-9062a9094fbb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:10.499326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc108d-ad23-4b0f-84d8-fc4edfc20fc5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:10.511502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:10.513620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:10.514680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:10.515049 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] PATH_INFO: `/v3/users/1268529f491a466882effe0456cccb1b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:10.517797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1268529f491a466882effe0456cccb1b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:10.520738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:10.528762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6d639-041e-4f98-9aaf-9062a9094fbb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:10.543809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39095d-bb6c-4edb-994f-2d08eabcebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:10.544787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39095d-bb6c-4edb-994f-2d08eabcebef admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:10.545208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39095d-bb6c-4edb-994f-2d08eabcebef admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:10.545599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39095d-bb6c-4edb-994f-2d08eabcebef admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:10.546522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39095d-bb6c-4edb-994f-2d08eabcebef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:10.550474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39095d-bb6c-4edb-994f-2d08eabcebef admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:10.599386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 898/1808] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:10 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:10.600891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8507b73-685c-46c0-b344-8cd8393ed9c4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:10.601292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8507b73-685c-46c0-b344-8cd8393ed9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:10.601640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8507b73-685c-46c0-b344-8cd8393ed9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:10.603190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1268529f491a466882effe0456cccb1b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:10.604084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1268529f491a466882effe0456cccb1b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:10.616625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1268529f491a466882effe0456cccb1b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:10.628001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:10.631366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1268529f491a466882effe0456cccb1b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:10.632343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1268529f491a466882effe0456cccb1b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:10.643087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d8507b73-685c-46c0-b344-8cd8393ed9c4 None None] MFA Rules not processed for user `c5c569cac8244c0ba2fe6dc3c9507e53`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:10.645181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d8507b73-685c-46c0-b344-8cd8393ed9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:10.661731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 911/1809] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:10 2022] DELETE /identity/v3/users/1268529f491a466882effe0456cccb1b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:10.662980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:10.665094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:10.679648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:10.768093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7714638-1e64-450d-bd84-91f1edf6cee0 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:10.772109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91caafda-79ea-4cf2-b01a-f0c0d5a99865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:10.772918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91caafda-79ea-4cf2-b01a-f0c0d5a99865 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:10.773250 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91caafda-79ea-4cf2-b01a-f0c0d5a99865 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:10.773588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91caafda-79ea-4cf2-b01a-f0c0d5a99865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:10.776693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-91caafda-79ea-4cf2-b01a-f0c0d5a99865 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:10.819444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 899/1810] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:10.819444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8507b73-685c-46c0-b344-8cd8393ed9c4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:10.824220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d8507b73-685c-46c0-b344-8cd8393ed9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:10.918319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91caafda-79ea-4cf2-b01a-f0c0d5a99865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:10.924242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91caafda-79ea-4cf2-b01a-f0c0d5a99865 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:10.949644 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 912/1811] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:10.950705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91caafda-79ea-4cf2-b01a-f0c0d5a99865 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:10.953970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-91caafda-79ea-4cf2-b01a-f0c0d5a99865 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:10.958504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8507b73-685c-46c0-b344-8cd8393ed9c4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:11.044833 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8c780-db8c-4d91-a562-957bd3127ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:11.046392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8c780-db8c-4d91-a562-957bd3127ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:11.046969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8c780-db8c-4d91-a562-957bd3127ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:11.047305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8c780-db8c-4d91-a562-957bd3127ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:11.050282 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6db8c780-db8c-4d91-a562-957bd3127ae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:11.053058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91caafda-79ea-4cf2-b01a-f0c0d5a99865 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:11.062108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0679284-bd45-46bc-bb82-e8b1362cec46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:11.063583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0679284-bd45-46bc-bb82-e8b1362cec46 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:11.064993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0679284-bd45-46bc-bb82-e8b1362cec46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:11.065382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0679284-bd45-46bc-bb82-e8b1362cec46 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:11.066235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0679284-bd45-46bc-bb82-e8b1362cec46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:11.066363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8c780-db8c-4d91-a562-957bd3127ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:11.067952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8c780-db8c-4d91-a562-957bd3127ae1 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:11.068737 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0679284-bd45-46bc-bb82-e8b1362cec46 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:11.098421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 900/1812] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:11.100464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8c780-db8c-4d91-a562-957bd3127ae1 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:11.101050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 913/1813] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:10 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:11.102893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0679284-bd45-46bc-bb82-e8b1362cec46 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:11.107890 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f0679284-bd45-46bc-bb82-e8b1362cec46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:11.108264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6db8c780-db8c-4d91-a562-957bd3127ae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:11.132950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0679284-bd45-46bc-bb82-e8b1362cec46 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:11.140414 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb3470-7c95-4043-9606-e74aa47f9f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:11.141266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb3470-7c95-4043-9606-e74aa47f9f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:11.143298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb3470-7c95-4043-9606-e74aa47f9f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:11.143742 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb3470-7c95-4043-9606-e74aa47f9f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:11.145054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb3470-7c95-4043-9606-e74aa47f9f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:11.147601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb3470-7c95-4043-9606-e74aa47f9f69 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:11.226387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 914/1814] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:11 2022] POST /identity/v3/projects => generated 363 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:11.227450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:11.230210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb3470-7c95-4043-9606-e74aa47f9f69 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:11.249765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8c780-db8c-4d91-a562-957bd3127ae1 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:11.316757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:11.374053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb3470-7c95-4043-9606-e74aa47f9f69 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:11.380580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc485f-5b3b-425b-9832-b510ff699daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:11.381721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc485f-5b3b-425b-9832-b510ff699daa admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:11.382125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc485f-5b3b-425b-9832-b510ff699daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:11.383668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc485f-5b3b-425b-9832-b510ff699daa admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:11.389525 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc485f-5b3b-425b-9832-b510ff699daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:11.392226 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc485f-5b3b-425b-9832-b510ff699daa admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:11.401957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2148b-f064-4ad6-a1be-14eab326ab1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:11.402811 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2148b-f064-4ad6-a1be-14eab326ab1b service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:11.403101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2148b-f064-4ad6-a1be-14eab326ab1b service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:11.403469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2148b-f064-4ad6-a1be-14eab326ab1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:11.405718 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-70b2148b-f064-4ad6-a1be-14eab326ab1b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:11.420864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 915/1815] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:11.422807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc485f-5b3b-425b-9832-b510ff699daa admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:11.444811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc485f-5b3b-425b-9832-b510ff699daa admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:11.458719 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57311962-eac0-4762-ae98-403927f5ca7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:11.461662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57311962-eac0-4762-ae98-403927f5ca7a admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:11.462063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57311962-eac0-4762-ae98-403927f5ca7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:11.462700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57311962-eac0-4762-ae98-403927f5ca7a admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:11.464950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57311962-eac0-4762-ae98-403927f5ca7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:11.469532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57311962-eac0-4762-ae98-403927f5ca7a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:11.485480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2148b-f064-4ad6-a1be-14eab326ab1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:11.487355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2148b-f064-4ad6-a1be-14eab326ab1b service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:11.497857 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 901/1816] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:15:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:15:11.498908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2148b-f064-4ad6-a1be-14eab326ab1b service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:11.511409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2148b-f064-4ad6-a1be-14eab326ab1b service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:11.517997 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a77734-3bdd-4a6d-ac09-fd111711d8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:11.518841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a77734-3bdd-4a6d-ac09-fd111711d8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:11.519200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a77734-3bdd-4a6d-ac09-fd111711d8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:11.519547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a77734-3bdd-4a6d-ac09-fd111711d8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:11.520388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a77734-3bdd-4a6d-ac09-fd111711d8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:11.522290 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a77734-3bdd-4a6d-ac09-fd111711d8d3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:11.526912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 916/1817] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:11 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:11.531385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57311962-eac0-4762-ae98-403927f5ca7a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:11.548567 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 902/1818] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:11.550390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a77734-3bdd-4a6d-ac09-fd111711d8d3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:11.598919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57311962-eac0-4762-ae98-403927f5ca7a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:11.601750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a77734-3bdd-4a6d-ac09-fd111711d8d3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:11.615604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0cd26-585f-48d9-a188-f5683902f379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:11.618168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0cd26-585f-48d9-a188-f5683902f379 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:11.618462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0cd26-585f-48d9-a188-f5683902f379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:11.618749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0cd26-585f-48d9-a188-f5683902f379 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:11.619387 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3060ec-7ee6-49f8-89ba-dc8f5eb2679f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:11.620771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3060ec-7ee6-49f8-89ba-dc8f5eb2679f admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:11.621130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3060ec-7ee6-49f8-89ba-dc8f5eb2679f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:11.621554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3060ec-7ee6-49f8-89ba-dc8f5eb2679f admin admin] PATH_INFO: `/v3/projects/20ce552af8eb4077b9971db6d235ed65/users/6877b13897804aa48c58bed4f7f80b81/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:11.623755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0cd26-585f-48d9-a188-f5683902f379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:11.629181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0cd26-585f-48d9-a188-f5683902f379 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:11.661798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3060ec-7ee6-49f8-89ba-dc8f5eb2679f admin admin] RBAC: Authorizing `identity:create_grant(project_id=20ce552af8eb4077b9971db6d235ed65, user_id=6877b13897804aa48c58bed4f7f80b81, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:11.667868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3060ec-7ee6-49f8-89ba-dc8f5eb2679f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:11.685586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 917/1819] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:11.689696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0cd26-585f-48d9-a188-f5683902f379 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:11.704315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 903/1820] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:11 2022] PUT /identity/v3/projects/20ce552af8eb4077b9971db6d235ed65/users/6877b13897804aa48c58bed4f7f80b81/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:11.707020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0cd26-585f-48d9-a188-f5683902f379 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:11.716724 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b58038-9aaf-4b93-b675-756db1a9b9fc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:11.717169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b58038-9aaf-4b93-b675-756db1a9b9fc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:11.717373 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b58038-9aaf-4b93-b675-756db1a9b9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:11.879457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d3b58038-9aaf-4b93-b675-756db1a9b9fc None None] MFA Rules not processed for user `6877b13897804aa48c58bed4f7f80b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:11.880870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d3b58038-9aaf-4b93-b675-756db1a9b9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:11.977293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799eaf6-0354-4e8a-b603-64aa8c0dfa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:11.978453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799eaf6-0354-4e8a-b603-64aa8c0dfa72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:11.978849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799eaf6-0354-4e8a-b603-64aa8c0dfa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:11.979237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799eaf6-0354-4e8a-b603-64aa8c0dfa72 admin admin] PATH_INFO: `/v3/projects/d7cda85462a2454db7c474c7c9c23283/users/73c4f136b420431fa676d4f2d510e802/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:12.070845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799eaf6-0354-4e8a-b603-64aa8c0dfa72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7cda85462a2454db7c474c7c9c23283, user_id=73c4f136b420431fa676d4f2d510e802, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:12.079844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799eaf6-0354-4e8a-b603-64aa8c0dfa72 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:12.097403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:12.123814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 904/1821] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:12.143929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 918/1822] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:11 2022] PUT /identity/v3/projects/d7cda85462a2454db7c474c7c9c23283/users/73c4f136b420431fa676d4f2d510e802/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:12.148123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799eaf6-0354-4e8a-b603-64aa8c0dfa72 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:12.161274 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b58038-9aaf-4b93-b675-756db1a9b9fc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:12.219184 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b58038-9aaf-4b93-b675-756db1a9b9fc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:12.353152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799eaf6-0354-4e8a-b603-64aa8c0dfa72 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:12.372356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296bfc1-114a-4b3e-86f1-45a09d815d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:12.373174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296bfc1-114a-4b3e-86f1-45a09d815d2e admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:12.373525 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296bfc1-114a-4b3e-86f1-45a09d815d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:12.373866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296bfc1-114a-4b3e-86f1-45a09d815d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:12.376035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296bfc1-114a-4b3e-86f1-45a09d815d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:12.376166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f4dab-06f0-4e09-8b9c-275e5884403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:12.376682 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f4dab-06f0-4e09-8b9c-275e5884403c admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:12.377198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296bfc1-114a-4b3e-86f1-45a09d815d2e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:12.377305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f4dab-06f0-4e09-8b9c-275e5884403c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:12.377815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f4dab-06f0-4e09-8b9c-275e5884403c admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:12.379579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f4dab-06f0-4e09-8b9c-275e5884403c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:12.382966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f4dab-06f0-4e09-8b9c-275e5884403c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:12.400794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 919/1823] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:15:12.402253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296bfc1-114a-4b3e-86f1-45a09d815d2e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:12.417359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d296bfc1-114a-4b3e-86f1-45a09d815d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:12.440744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 905/1824] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:12 2022] POST /identity/v3/projects => generated 381 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:12.442323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f4dab-06f0-4e09-8b9c-275e5884403c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:12.454953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:12.583888 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296bfc1-114a-4b3e-86f1-45a09d815d2e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:12.600646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f4dab-06f0-4e09-8b9c-275e5884403c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:12.614025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21362537-2944-404b-a11e-84b9fcf33495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:12.614988 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21362537-2944-404b-a11e-84b9fcf33495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:12.615502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21362537-2944-404b-a11e-84b9fcf33495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:12.615806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21362537-2944-404b-a11e-84b9fcf33495 admin admin] PATH_INFO: `/v3/projects/d7cda85462a2454db7c474c7c9c23283/users/73c4f136b420431fa676d4f2d510e802/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:12.621037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21362537-2944-404b-a11e-84b9fcf33495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7cda85462a2454db7c474c7c9c23283, user_id=73c4f136b420431fa676d4f2d510e802, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:12.629017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21362537-2944-404b-a11e-84b9fcf33495 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:12.658758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 906/1825] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:12 2022] PUT /identity/v3/projects/d7cda85462a2454db7c474c7c9c23283/users/73c4f136b420431fa676d4f2d510e802/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:12.659662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21362537-2944-404b-a11e-84b9fcf33495 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:12.714717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2508ce-14a3-4701-a36f-81a5ac9b24be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:12.715440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2508ce-14a3-4701-a36f-81a5ac9b24be service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:12.716119 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2508ce-14a3-4701-a36f-81a5ac9b24be service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:12.716410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2508ce-14a3-4701-a36f-81a5ac9b24be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:12.719390 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-dc2508ce-14a3-4701-a36f-81a5ac9b24be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:12.957866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2508ce-14a3-4701-a36f-81a5ac9b24be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:12.964895 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2508ce-14a3-4701-a36f-81a5ac9b24be service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:12.992500 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 920/1826] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:13.008803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bd0da-8d98-4401-875a-7a0dc631b9ef None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:13.009329 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bd0da-8d98-4401-875a-7a0dc631b9ef None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:13.010628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bd0da-8d98-4401-875a-7a0dc631b9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:13.080867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-291bd0da-8d98-4401-875a-7a0dc631b9ef None None] MFA Rules not processed for user `73c4f136b420431fa676d4f2d510e802`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:13.082453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-291bd0da-8d98-4401-875a-7a0dc631b9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:13.207330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21362537-2944-404b-a11e-84b9fcf33495 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:13.218122 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308297e-73a6-425b-a02b-9c5fe4ffa911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:13.220822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308297e-73a6-425b-a02b-9c5fe4ffa911 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:13.221244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308297e-73a6-425b-a02b-9c5fe4ffa911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:13.221552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308297e-73a6-425b-a02b-9c5fe4ffa911 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:13.224807 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308297e-73a6-425b-a02b-9c5fe4ffa911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:13.225746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308297e-73a6-425b-a02b-9c5fe4ffa911 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:13.230349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 921/1827] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:12 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:13.234141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bd0da-8d98-4401-875a-7a0dc631b9ef None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:13.239139 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-291bd0da-8d98-4401-875a-7a0dc631b9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:13.290367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bd0da-8d98-4401-875a-7a0dc631b9ef None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:13.295182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 907/1828] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:12 2022] POST /identity/v3/users => generated 297 bytes in 635 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:13.296756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308297e-73a6-425b-a02b-9c5fe4ffa911 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:13.313198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:13.314866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:13.317449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:13.319879 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:13.320089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] PATH_INFO: `/v3/projects/eb49ab75fe7d434c95d639cbd0bcb634` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:13.324437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb49ab75fe7d434c95d639cbd0bcb634)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:13.337068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:13.352892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:13.458911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308297e-73a6-425b-a02b-9c5fe4ffa911 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:13.467121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb49ab75fe7d434c95d639cbd0bcb634', 'request_id': 'req-c1dd8872-c06d-4b82-bf00-b246094bf9e1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:13.483474 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb49ab75fe7d434c95d639cbd0bcb634', 'request_id': 'req-c1dd8872-c06d-4b82-bf00-b246094bf9e1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:13.493195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb49ab75fe7d434c95d639cbd0bcb634', 'request_id': 'req-c1dd8872-c06d-4b82-bf00-b246094bf9e1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:13.529759 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] The token cache is being invalidate because project eb49ab75fe7d434c95d639cbd0bcb634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:13.530200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:13.531172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:13.533545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 922/1829] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:13 2022] DELETE /identity/v3/projects/eb49ab75fe7d434c95d639cbd0bcb634 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:13.534297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:13.536043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:13.584026 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab89ea-c652-400d-a123-5a84e6aa7c87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:13.595939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab89ea-c652-400d-a123-5a84e6aa7c87 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:13.596248 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab89ea-c652-400d-a123-5a84e6aa7c87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:13.596484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab89ea-c652-400d-a123-5a84e6aa7c87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:13.599417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-01ab89ea-c652-400d-a123-5a84e6aa7c87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:13.639078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd8872-c06d-4b82-bf00-b246094bf9e1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:13.644136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf80f2-8f72-4205-9a54-e88885623ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:13.645004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf80f2-8f72-4205-9a54-e88885623ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:13.645299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf80f2-8f72-4205-9a54-e88885623ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:13.645812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf80f2-8f72-4205-9a54-e88885623ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:13.646618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf80f2-8f72-4205-9a54-e88885623ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:13.655750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf80f2-8f72-4205-9a54-e88885623ab0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:13.674748 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 923/1830] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:13.676663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28c53a-8efd-4e9d-ad4d-c1ae922ccf85 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:13.676844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28c53a-8efd-4e9d-ad4d-c1ae922ccf85 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:13.677117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28c53a-8efd-4e9d-ad4d-c1ae922ccf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:13.696929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2b28c53a-8efd-4e9d-ad4d-c1ae922ccf85 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:13.698256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2b28c53a-8efd-4e9d-ad4d-c1ae922ccf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:13.719051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 924/1831] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:13.720062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28c53a-8efd-4e9d-ad4d-c1ae922ccf85 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:13.731341 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28c53a-8efd-4e9d-ad4d-c1ae922ccf85 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:13.737503 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671506db-5da9-435f-866b-caacadccf6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:13.738570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671506db-5da9-435f-866b-caacadccf6b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:13.739061 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671506db-5da9-435f-866b-caacadccf6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:13.739836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671506db-5da9-435f-866b-caacadccf6b4 admin admin] PATH_INFO: `/v3/projects/33e5c18fc2164ce1b2ec6ab10781a99c/users/056f86dc774e451ca3d140e2cb708d3a/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:13.774299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671506db-5da9-435f-866b-caacadccf6b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e5c18fc2164ce1b2ec6ab10781a99c, user_id=056f86dc774e451ca3d140e2cb708d3a, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:13.777509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671506db-5da9-435f-866b-caacadccf6b4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:13.786885 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab89ea-c652-400d-a123-5a84e6aa7c87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:13.790180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab89ea-c652-400d-a123-5a84e6aa7c87 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:13.802433 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 925/1832] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:13 2022] PUT /identity/v3/projects/33e5c18fc2164ce1b2ec6ab10781a99c/users/056f86dc774e451ca3d140e2cb708d3a/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:13.802654 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34574c0b-4dbb-4180-952a-d06141397dc1 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:13.803007 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34574c0b-4dbb-4180-952a-d06141397dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:13.803383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34574c0b-4dbb-4180-952a-d06141397dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:13.838859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-34574c0b-4dbb-4180-952a-d06141397dc1 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:13.840317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-34574c0b-4dbb-4180-952a-d06141397dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:13.855393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 908/1833] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:13 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:13.857736 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab89ea-c652-400d-a123-5a84e6aa7c87 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:13.893618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab89ea-c652-400d-a123-5a84e6aa7c87 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:13.927576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:13.961608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 926/1834] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:13.963870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34574c0b-4dbb-4180-952a-d06141397dc1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:13.986426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34574c0b-4dbb-4180-952a-d06141397dc1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:13.993485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8d491-268c-4189-aa98-cfb5fdfb112a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:13.994360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8d491-268c-4189-aa98-cfb5fdfb112a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:13.994747 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8d491-268c-4189-aa98-cfb5fdfb112a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:13.995173 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8d491-268c-4189-aa98-cfb5fdfb112a admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:13.996023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8d491-268c-4189-aa98-cfb5fdfb112a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:13.996435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8d491-268c-4189-aa98-cfb5fdfb112a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:13.998920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8d491-268c-4189-aa98-cfb5fdfb112a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:14.021226 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 927/1835] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:14.025922 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8d491-268c-4189-aa98-cfb5fdfb112a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:14.028354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1080fe-799e-452f-bc58-5ddd686541a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:14.029418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1080fe-799e-452f-bc58-5ddd686541a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.029706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1080fe-799e-452f-bc58-5ddd686541a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.029926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5bc8d491-268c-4189-aa98-cfb5fdfb112a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:14.030032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1080fe-799e-452f-bc58-5ddd686541a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.030751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1080fe-799e-452f-bc58-5ddd686541a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:14.032671 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1080fe-799e-452f-bc58-5ddd686541a3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:14.048808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 909/1836] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:14.049590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1080fe-799e-452f-bc58-5ddd686541a3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:14.060691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1080fe-799e-452f-bc58-5ddd686541a3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:14.069413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db3964-afcc-426c-bcb5-91380051a219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:14.070076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db3964-afcc-426c-bcb5-91380051a219 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.070365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db3964-afcc-426c-bcb5-91380051a219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.070645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db3964-afcc-426c-bcb5-91380051a219 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.071363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db3964-afcc-426c-bcb5-91380051a219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:14.076151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db3964-afcc-426c-bcb5-91380051a219 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:14.085741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:14.147139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 910/1837] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:14 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:14.151571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db3964-afcc-426c-bcb5-91380051a219 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:14.162731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8d491-268c-4189-aa98-cfb5fdfb112a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:14.284682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038977b-79ef-425f-bddf-1ccc26033435 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:14.285551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038977b-79ef-425f-bddf-1ccc26033435 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.286078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038977b-79ef-425f-bddf-1ccc26033435 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.290199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038977b-79ef-425f-bddf-1ccc26033435 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.292828 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3038977b-79ef-425f-bddf-1ccc26033435 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:14.295064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db3964-afcc-426c-bcb5-91380051a219 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:14.300847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96497c5f-dfa3-49e0-b8fd-a6a4ce8b059c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:14.301984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96497c5f-dfa3-49e0-b8fd-a6a4ce8b059c admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.302744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96497c5f-dfa3-49e0-b8fd-a6a4ce8b059c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.302744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96497c5f-dfa3-49e0-b8fd-a6a4ce8b059c admin admin] PATH_INFO: `/v3/projects/33e5c18fc2164ce1b2ec6ab10781a99c/users/056f86dc774e451ca3d140e2cb708d3a/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.306124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96497c5f-dfa3-49e0-b8fd-a6a4ce8b059c admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e5c18fc2164ce1b2ec6ab10781a99c, user_id=056f86dc774e451ca3d140e2cb708d3a, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:14.314009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96497c5f-dfa3-49e0-b8fd-a6a4ce8b059c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:14.336075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 911/1838] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:14 2022] PUT /identity/v3/projects/33e5c18fc2164ce1b2ec6ab10781a99c/users/056f86dc774e451ca3d140e2cb708d3a/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:14.336836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96497c5f-dfa3-49e0-b8fd-a6a4ce8b059c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:14.400189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038977b-79ef-425f-bddf-1ccc26033435 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:14.403944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038977b-79ef-425f-bddf-1ccc26033435 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:14.422432 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 928/1839] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:14.423998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c983b-cc08-4acb-9322-6d81079a1cd1 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.424474 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c983b-cc08-4acb-9322-6d81079a1cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.425011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c983b-cc08-4acb-9322-6d81079a1cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.437928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96497c5f-dfa3-49e0-b8fd-a6a4ce8b059c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:14.442419 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181cab3-b262-4254-bec2-1ff48e2d7244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:14.443203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cab3-b262-4254-bec2-1ff48e2d7244 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.443587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cab3-b262-4254-bec2-1ff48e2d7244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.443960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cab3-b262-4254-bec2-1ff48e2d7244 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.444779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181cab3-b262-4254-bec2-1ff48e2d7244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:14.446996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181cab3-b262-4254-bec2-1ff48e2d7244 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:14.474769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f45c983b-cc08-4acb-9322-6d81079a1cd1 None None] MFA Rules not processed for user `056f86dc774e451ca3d140e2cb708d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:14.476062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f45c983b-cc08-4acb-9322-6d81079a1cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:14.479575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 912/1840] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:14 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:14.481741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e955a9c-d27b-4b00-a510-f7d6f2069e97 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.482324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e955a9c-d27b-4b00-a510-f7d6f2069e97 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.482787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e955a9c-d27b-4b00-a510-f7d6f2069e97 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.484593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 913/1841] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:15:14.485586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e955a9c-d27b-4b00-a510-f7d6f2069e97 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:14.514018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e955a9c-d27b-4b00-a510-f7d6f2069e97 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:14.527496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e8d35-e5e2-40e8-9d79-6d59a4ac8a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:14.529115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e8d35-e5e2-40e8-9d79-6d59a4ac8a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.530421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e8d35-e5e2-40e8-9d79-6d59a4ac8a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.530994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e8d35-e5e2-40e8-9d79-6d59a4ac8a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.531903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e8d35-e5e2-40e8-9d79-6d59a4ac8a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:14.534693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e8d35-e5e2-40e8-9d79-6d59a4ac8a33 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:14.552409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 914/1842] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:14.552409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9a8f0-2e38-454d-bd7b-e0b0d29948b6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.552409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9a8f0-2e38-454d-bd7b-e0b0d29948b6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.552409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9a8f0-2e38-454d-bd7b-e0b0d29948b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.560191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:14.582663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cae9a8f0-2e38-454d-bd7b-e0b0d29948b6 None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:14.584073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cae9a8f0-2e38-454d-bd7b-e0b0d29948b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:14.597862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 929/1843] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:14.599144 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c983b-cc08-4acb-9322-6d81079a1cd1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:14.633723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c983b-cc08-4acb-9322-6d81079a1cd1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:14.641122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff9bd-6d48-449f-bde9-9187b17deffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:14.641829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff9bd-6d48-449f-bde9-9187b17deffe admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.642164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff9bd-6d48-449f-bde9-9187b17deffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.642443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff9bd-6d48-449f-bde9-9187b17deffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.643187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bff9bd-6d48-449f-bde9-9187b17deffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:14.645321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bff9bd-6d48-449f-bde9-9187b17deffe admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:14.658184 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 930/1844] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:14.659007 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff9bd-6d48-449f-bde9-9187b17deffe admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:14.660830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-25bff9bd-6d48-449f-bde9-9187b17deffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:14.662475 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:14.687756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 915/1845] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:15:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:14.688593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9a8f0-2e38-454d-bd7b-e0b0d29948b6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:14.698881 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9a8f0-2e38-454d-bd7b-e0b0d29948b6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:14.706315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000d20e-a820-4de3-81c6-edc54a54f536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:14.707046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000d20e-a820-4de3-81c6-edc54a54f536 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.707550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000d20e-a820-4de3-81c6-edc54a54f536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.707828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000d20e-a820-4de3-81c6-edc54a54f536 admin admin] PATH_INFO: `/v3/projects/44af3132624249729b6150d85a707f23/users/e0c6f5504b304d6f86794e0110880a5c/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.724823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000d20e-a820-4de3-81c6-edc54a54f536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44af3132624249729b6150d85a707f23, user_id=e0c6f5504b304d6f86794e0110880a5c, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:14.727572 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000d20e-a820-4de3-81c6-edc54a54f536 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:14.747917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 916/1846] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:14 2022] PUT /identity/v3/projects/44af3132624249729b6150d85a707f23/users/e0c6f5504b304d6f86794e0110880a5c/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:14.748802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000d20e-a820-4de3-81c6-edc54a54f536 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:14.749543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff9bd-6d48-449f-bde9-9187b17deffe admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:14.750503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3000d20e-a820-4de3-81c6-edc54a54f536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:14.801801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7971130-e8f9-45fc-bd96-1bbc733748cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:14.802531 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7971130-e8f9-45fc-bd96-1bbc733748cd service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.802814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7971130-e8f9-45fc-bd96-1bbc733748cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.803088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7971130-e8f9-45fc-bd96-1bbc733748cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.828669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000d20e-a820-4de3-81c6-edc54a54f536 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:14.835837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12fbc4-f8a6-48c6-9c94-683e0256ef73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:14.835837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12fbc4-f8a6-48c6-9c94-683e0256ef73 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.836130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12fbc4-f8a6-48c6-9c94-683e0256ef73 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.836697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12fbc4-f8a6-48c6-9c94-683e0256ef73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.846751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12fbc4-f8a6-48c6-9c94-683e0256ef73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:14.848344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12fbc4-f8a6-48c6-9c94-683e0256ef73 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:14.860465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 917/1847] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:14.861573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7e4f6-76b8-4e3d-9736-33f5d3c4ba30 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:14.861900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7e4f6-76b8-4e3d-9736-33f5d3c4ba30 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:14.862199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7e4f6-76b8-4e3d-9736-33f5d3c4ba30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:14.879599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7971130-e8f9-45fc-bd96-1bbc733748cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:14.881069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7971130-e8f9-45fc-bd96-1bbc733748cd service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:14.891482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f3f7e4f6-76b8-4e3d-9736-33f5d3c4ba30 None None] MFA Rules not processed for user `e0c6f5504b304d6f86794e0110880a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:14.892762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f3f7e4f6-76b8-4e3d-9736-33f5d3c4ba30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:14.898905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 931/1848] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:14.978088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 918/1849] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:14.982777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7971130-e8f9-45fc-bd96-1bbc733748cd service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:15.019516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7e4f6-76b8-4e3d-9736-33f5d3c4ba30 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:15.030011 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7e4f6-76b8-4e3d-9736-33f5d3c4ba30 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:15.033968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b26cc-3072-4851-a7f6-4cceb58a9706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:15.034629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b26cc-3072-4851-a7f6-4cceb58a9706 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:15.034909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b26cc-3072-4851-a7f6-4cceb58a9706 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:15.035181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b26cc-3072-4851-a7f6-4cceb58a9706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:15.036310 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:15.037477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a67b26cc-3072-4851-a7f6-4cceb58a9706 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:15.070759 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7971130-e8f9-45fc-bd96-1bbc733748cd service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:15.077169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9041808-3a63-48c6-aae4-5c84539b7618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:15.077843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9041808-3a63-48c6-aae4-5c84539b7618 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:15.078193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9041808-3a63-48c6-aae4-5c84539b7618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:15.078471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9041808-3a63-48c6-aae4-5c84539b7618 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:15.079166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9041808-3a63-48c6-aae4-5c84539b7618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:15.082071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9041808-3a63-48c6-aae4-5c84539b7618 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:15.099189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:15.117466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 932/1850] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:14 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:15:15.121604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9041808-3a63-48c6-aae4-5c84539b7618 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:15.126329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b26cc-3072-4851-a7f6-4cceb58a9706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:15.128015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b26cc-3072-4851-a7f6-4cceb58a9706 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:15.139104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 919/1851] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:15:15.220024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9041808-3a63-48c6-aae4-5c84539b7618 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:15.227847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a231f36-51e4-40e4-989f-84a0604c3516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:15.228535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a231f36-51e4-40e4-989f-84a0604c3516 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:15.229005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a231f36-51e4-40e4-989f-84a0604c3516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:15.229420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a231f36-51e4-40e4-989f-84a0604c3516 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:15.230587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a231f36-51e4-40e4-989f-84a0604c3516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:15.233492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a231f36-51e4-40e4-989f-84a0604c3516 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:15.268869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 933/1852] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:15 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:15:15.273765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b26cc-3072-4851-a7f6-4cceb58a9706 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:15.289742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b26cc-3072-4851-a7f6-4cceb58a9706 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:15.322447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5289dc-6aec-4003-8b1d-92aac4794a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:15.324319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5289dc-6aec-4003-8b1d-92aac4794a72 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:15.325098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5289dc-6aec-4003-8b1d-92aac4794a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:15.325729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5289dc-6aec-4003-8b1d-92aac4794a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:15.328476 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5289dc-6aec-4003-8b1d-92aac4794a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:15.332174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5289dc-6aec-4003-8b1d-92aac4794a72 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:15.362544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 920/1853] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:15.365946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a231f36-51e4-40e4-989f-84a0604c3516 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:15.380999 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a231f36-51e4-40e4-989f-84a0604c3516 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:15.387342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32af2b-738c-478f-bbd3-d834d0cd2964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:15.388474 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32af2b-738c-478f-bbd3-d834d0cd2964 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:15.388781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32af2b-738c-478f-bbd3-d834d0cd2964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:15.389157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32af2b-738c-478f-bbd3-d834d0cd2964 admin admin] PATH_INFO: `/v3/projects/f2118e40506844a8bb83c010c520689d/users/997657bd8b8646e19defff8bd43bbde1/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:15.423771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32af2b-738c-478f-bbd3-d834d0cd2964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2118e40506844a8bb83c010c520689d, user_id=997657bd8b8646e19defff8bd43bbde1, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:15.426962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32af2b-738c-478f-bbd3-d834d0cd2964 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:15.446462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 934/1854] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:15 2022] PUT /identity/v3/projects/f2118e40506844a8bb83c010c520689d/users/997657bd8b8646e19defff8bd43bbde1/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:15.450443 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5289dc-6aec-4003-8b1d-92aac4794a72 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:15.465100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5289dc-6aec-4003-8b1d-92aac4794a72 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:15.572892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0cca4-dccb-42d4-b08d-ae499243613b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:15.573720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0cca4-dccb-42d4-b08d-ae499243613b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:15.574215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0cca4-dccb-42d4-b08d-ae499243613b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:15.574496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0cca4-dccb-42d4-b08d-ae499243613b admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:15.575243 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0cca4-dccb-42d4-b08d-ae499243613b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:15.576871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0cca4-dccb-42d4-b08d-ae499243613b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:15.587713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 921/1855] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:15.591976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32af2b-738c-478f-bbd3-d834d0cd2964 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:15.604148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32af2b-738c-478f-bbd3-d834d0cd2964 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:15.609419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12485614-526b-475f-a7c5-16bc4b2ad142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:15.610496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12485614-526b-475f-a7c5-16bc4b2ad142 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:15.611023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12485614-526b-475f-a7c5-16bc4b2ad142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:15.611370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12485614-526b-475f-a7c5-16bc4b2ad142 admin admin] PATH_INFO: `/v3/projects/f2118e40506844a8bb83c010c520689d/users/997657bd8b8646e19defff8bd43bbde1/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:15.613511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12485614-526b-475f-a7c5-16bc4b2ad142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2118e40506844a8bb83c010c520689d, user_id=997657bd8b8646e19defff8bd43bbde1, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:15.616442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12485614-526b-475f-a7c5-16bc4b2ad142 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:15.627954 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 935/1856] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:15 2022] PUT /identity/v3/projects/f2118e40506844a8bb83c010c520689d/users/997657bd8b8646e19defff8bd43bbde1/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:15.639447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580b986-100b-4b92-bc5c-fa501ac41e73 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:15.639749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580b986-100b-4b92-bc5c-fa501ac41e73 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:15.640038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580b986-100b-4b92-bc5c-fa501ac41e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:15.681685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f580b986-100b-4b92-bc5c-fa501ac41e73 None None] MFA Rules not processed for user `997657bd8b8646e19defff8bd43bbde1`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:15.683551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f580b986-100b-4b92-bc5c-fa501ac41e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:15.775072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 922/1857] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:15.829000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12485614-526b-475f-a7c5-16bc4b2ad142 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:15.908293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12485614-526b-475f-a7c5-16bc4b2ad142 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:15.913146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c81543-d44b-4d99-bd74-1d5fe6383977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:15.913644 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c81543-d44b-4d99-bd74-1d5fe6383977 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:15.913933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c81543-d44b-4d99-bd74-1d5fe6383977 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:15.914209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c81543-d44b-4d99-bd74-1d5fe6383977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:15.925151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c81543-d44b-4d99-bd74-1d5fe6383977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:15.926728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c81543-d44b-4d99-bd74-1d5fe6383977 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:15.947977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 936/1858] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:16.095080 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580b986-100b-4b92-bc5c-fa501ac41e73 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:16.110428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580b986-100b-4b92-bc5c-fa501ac41e73 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:16.130071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715756ff-5cf8-46a0-921d-4fb180ee66b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:16.130071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715756ff-5cf8-46a0-921d-4fb180ee66b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:16.130071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715756ff-5cf8-46a0-921d-4fb180ee66b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:16.130071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715756ff-5cf8-46a0-921d-4fb180ee66b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:16.267759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:16.322521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715756ff-5cf8-46a0-921d-4fb180ee66b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:16.324273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715756ff-5cf8-46a0-921d-4fb180ee66b8 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:16.341196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 923/1859] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:18.128379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c81543-d44b-4d99-bd74-1d5fe6383977 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:18.133691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-11c81543-d44b-4d99-bd74-1d5fe6383977 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:18.175218 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:18.307753 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c81543-d44b-4d99-bd74-1d5fe6383977 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:18.316783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:18.317695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:18.318152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:18.318394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] PATH_INFO: `/v3/users/c5c569cac8244c0ba2fe6dc3c9507e53` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:18.319809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c569cac8244c0ba2fe6dc3c9507e53)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:18.323327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:18.419877 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c569cac8244c0ba2fe6dc3c9507e53'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:18.424605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c569cac8244c0ba2fe6dc3c9507e53'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:18.442799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c569cac8244c0ba2fe6dc3c9507e53'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:18.459645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c569cac8244c0ba2fe6dc3c9507e53'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:18.464613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c569cac8244c0ba2fe6dc3c9507e53'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:18.518631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 937/1860] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:18 2022] DELETE /identity/v3/users/c5c569cac8244c0ba2fe6dc3c9507e53 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:18.524055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ffab9-84e5-4ca9-84b1-70d32ab503c8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:18.524628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ffab9-84e5-4ca9-84b1-70d32ab503c8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:18.524985 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ffab9-84e5-4ca9-84b1-70d32ab503c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:18.561482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1a2ffab9-84e5-4ca9-84b1-70d32ab503c8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:18.567147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1a2ffab9-84e5-4ca9-84b1-70d32ab503c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:18.750159 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 924/1861] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:18.824427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:18.826036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:18.970930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb2b45-f8b7-4fa1-b2da-a5b19a080306 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:19.003564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ffab9-84e5-4ca9-84b1-70d32ab503c8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:19.005321 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1a2ffab9-84e5-4ca9-84b1-70d32ab503c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:19.007438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad84ef-f4b3-4644-850e-bff3a799a314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:19.008762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad84ef-f4b3-4644-850e-bff3a799a314 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.009320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad84ef-f4b3-4644-850e-bff3a799a314 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.009807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad84ef-f4b3-4644-850e-bff3a799a314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.024876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad84ef-f4b3-4644-850e-bff3a799a314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:19.027330 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad84ef-f4b3-4644-850e-bff3a799a314 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:19.054631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 938/1862] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:19.102769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ffab9-84e5-4ca9-84b1-70d32ab503c8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:19.106613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbc03f-4785-4f8c-9bd3-320a9e97681a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:19.109587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbc03f-4785-4f8c-9bd3-320a9e97681a service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:19.109871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbc03f-4785-4f8c-9bd3-320a9e97681a service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:19.110185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbc03f-4785-4f8c-9bd3-320a9e97681a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:19.112549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f9cbc03f-4785-4f8c-9bd3-320a9e97681a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:19.201914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbc03f-4785-4f8c-9bd3-320a9e97681a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:19.204842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbc03f-4785-4f8c-9bd3-320a9e97681a service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:19.235578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 925/1863] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:20.446645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d2357-ad50-4e2d-b90f-9fd1e9b9df12 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:20.447688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d2357-ad50-4e2d-b90f-9fd1e9b9df12 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:20.448583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d2357-ad50-4e2d-b90f-9fd1e9b9df12 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:20.450705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 939/1864] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:15:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:15:20.464291 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d5e6-792e-490f-ab30-dc09ca8855c3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:20.464928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d5e6-792e-490f-ab30-dc09ca8855c3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:20.465110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d5e6-792e-490f-ab30-dc09ca8855c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:20.483043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d2357-ad50-4e2d-b90f-9fd1e9b9df12 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:20.485065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d77d2357-ad50-4e2d-b90f-9fd1e9b9df12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:20.492026 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-29c9d5e6-792e-490f-ab30-dc09ca8855c3 None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:20.493816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-29c9d5e6-792e-490f-ab30-dc09ca8855c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:20.509405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d2357-ad50-4e2d-b90f-9fd1e9b9df12 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:20.515757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:20.517378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:20.517378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:20.517640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] PATH_INFO: `/v3/projects/00fbe799557e40c0a8253f134abbdb74` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:20.519917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00fbe799557e40c0a8253f134abbdb74)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:20.522460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:20.548932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:20.574163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:20.581672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 926/1865] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:15:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:20.591331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00fbe799557e40c0a8253f134abbdb74', 'request_id': 'req-3ab0f03c-9340-4a3b-9615-c59aace29b56'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.594874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9d5e6-792e-490f-ab30-dc09ca8855c3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:20.596352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-29c9d5e6-792e-490f-ab30-dc09ca8855c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:20.601118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00fbe799557e40c0a8253f134abbdb74', 'request_id': 'req-3ab0f03c-9340-4a3b-9615-c59aace29b56'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.609749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00fbe799557e40c0a8253f134abbdb74', 'request_id': 'req-3ab0f03c-9340-4a3b-9615-c59aace29b56'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.611560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9d5e6-792e-490f-ab30-dc09ca8855c3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:20.620672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] The token cache is being invalidate because project 00fbe799557e40c0a8253f134abbdb74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:20.620962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.621790 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.624961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 940/1866] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:20 2022] DELETE /identity/v3/projects/00fbe799557e40c0a8253f134abbdb74 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:20.628614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:20.630045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:20.679375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af177a-266c-4d5f-a68e-51bf191b384d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:20.679960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af177a-266c-4d5f-a68e-51bf191b384d service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:20.680353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af177a-266c-4d5f-a68e-51bf191b384d service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:20.680633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af177a-266c-4d5f-a68e-51bf191b384d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:20.682559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b2af177a-266c-4d5f-a68e-51bf191b384d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:20.694485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af177a-266c-4d5f-a68e-51bf191b384d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:20.694713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0f03c-9340-4a3b-9615-c59aace29b56 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:20.696073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af177a-266c-4d5f-a68e-51bf191b384d service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:20.700035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef1487-6ebf-4c1c-b230-c85950fb0488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:20.701096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef1487-6ebf-4c1c-b230-c85950fb0488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:20.701380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef1487-6ebf-4c1c-b230-c85950fb0488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:20.701659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef1487-6ebf-4c1c-b230-c85950fb0488 admin admin] PATH_INFO: `/v3/users/73c4f136b420431fa676d4f2d510e802` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:20.702934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef1487-6ebf-4c1c-b230-c85950fb0488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c4f136b420431fa676d4f2d510e802)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:20.704906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef1487-6ebf-4c1c-b230-c85950fb0488 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:20.706888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 927/1867] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:20.754455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6eef1487-6ebf-4c1c-b230-c85950fb0488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c4f136b420431fa676d4f2d510e802'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.755105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6eef1487-6ebf-4c1c-b230-c85950fb0488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c4f136b420431fa676d4f2d510e802'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.764622 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6eef1487-6ebf-4c1c-b230-c85950fb0488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c4f136b420431fa676d4f2d510e802'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.771369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6eef1487-6ebf-4c1c-b230-c85950fb0488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c4f136b420431fa676d4f2d510e802'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.772016 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6eef1487-6ebf-4c1c-b230-c85950fb0488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c4f136b420431fa676d4f2d510e802'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:20.789763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 941/1868] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:20 2022] DELETE /identity/v3/users/73c4f136b420431fa676d4f2d510e802 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:21.102714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af177a-266c-4d5f-a68e-51bf191b384d service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:21.124685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af177a-266c-4d5f-a68e-51bf191b384d service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:21.290633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:21.291665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:21.292101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:21.292470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] PATH_INFO: `/v3/projects/d7cda85462a2454db7c474c7c9c23283` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:21.293782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7cda85462a2454db7c474c7c9c23283)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:21.295839 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:21.375862 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7cda85462a2454db7c474c7c9c23283', 'request_id': 'req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:21.390307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7cda85462a2454db7c474c7c9c23283', 'request_id': 'req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:21.408642 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7cda85462a2454db7c474c7c9c23283', 'request_id': 'req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:21.410276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:21.433260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] The token cache is being invalidate because project d7cda85462a2454db7c474c7c9c23283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:21.433552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:21.437351 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-33b2b8c5-56f1-46d3-92d7-d423fc8734f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:21.443478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 928/1869] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:21 2022] DELETE /identity/v3/projects/d7cda85462a2454db7c474c7c9c23283 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:21.449338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03761dc-1c8d-4821-b646-4cfbdac9eea6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:21.450131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03761dc-1c8d-4821-b646-4cfbdac9eea6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:21.450131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03761dc-1c8d-4821-b646-4cfbdac9eea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:21.501536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b03761dc-1c8d-4821-b646-4cfbdac9eea6 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:21.506121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b03761dc-1c8d-4821-b646-4cfbdac9eea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:21.583069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:21.638873 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 942/1870] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:21.647014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cfb00-e48b-4084-8700-14c38ed1fc0b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:21.647901 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cfb00-e48b-4084-8700-14c38ed1fc0b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:21.648322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cfb00-e48b-4084-8700-14c38ed1fc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:21.668339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-009cfb00-e48b-4084-8700-14c38ed1fc0b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:21.669851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-009cfb00-e48b-4084-8700-14c38ed1fc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:21.678733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 929/1871] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:21.682965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03761dc-1c8d-4821-b646-4cfbdac9eea6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:21.694342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03761dc-1c8d-4821-b646-4cfbdac9eea6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:21.699909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ad448-ad89-44e0-935b-dd2f09c2d9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:21.700689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ad448-ad89-44e0-935b-dd2f09c2d9ce admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:21.700974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ad448-ad89-44e0-935b-dd2f09c2d9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:21.701249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ad448-ad89-44e0-935b-dd2f09c2d9ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:21.702043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ad448-ad89-44e0-935b-dd2f09c2d9ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:21.702355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ad448-ad89-44e0-935b-dd2f09c2d9ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:21.703996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ad448-ad89-44e0-935b-dd2f09c2d9ce admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:21.720231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 943/1872] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:21.721368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cfb00-e48b-4084-8700-14c38ed1fc0b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:21.731210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cfb00-e48b-4084-8700-14c38ed1fc0b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:21.735871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0d19a-9305-4bec-871d-4df820193a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:21.736552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0d19a-9305-4bec-871d-4df820193a50 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:21.736838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0d19a-9305-4bec-871d-4df820193a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:21.737253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0d19a-9305-4bec-871d-4df820193a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:21.738032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0d19a-9305-4bec-871d-4df820193a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:21.740128 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0d19a-9305-4bec-871d-4df820193a50 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:21.776970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 930/1873] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:21 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:21.778332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ad448-ad89-44e0-935b-dd2f09c2d9ce admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:21.788820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ad448-ad89-44e0-935b-dd2f09c2d9ce admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:21.868150 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b87954-ead9-4338-b6cd-8f3a03def3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:21.869056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b87954-ead9-4338-b6cd-8f3a03def3af admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:21.869338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b87954-ead9-4338-b6cd-8f3a03def3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:21.869595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b87954-ead9-4338-b6cd-8f3a03def3af admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:21.870425 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b87954-ead9-4338-b6cd-8f3a03def3af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:21.872375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b87954-ead9-4338-b6cd-8f3a03def3af admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:21.906392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 944/1874] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:21 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:21.910685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0d19a-9305-4bec-871d-4df820193a50 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:21.921323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0d19a-9305-4bec-871d-4df820193a50 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:21.926210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce35a74-3547-4179-9a40-6090bba2bdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:21.926882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce35a74-3547-4179-9a40-6090bba2bdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:21.927165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce35a74-3547-4179-9a40-6090bba2bdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:21.927451 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce35a74-3547-4179-9a40-6090bba2bdb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:21.928212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce35a74-3547-4179-9a40-6090bba2bdb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:21.929915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce35a74-3547-4179-9a40-6090bba2bdb3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:21.946346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 931/1875] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:21.951261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b87954-ead9-4338-b6cd-8f3a03def3af admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:21.962893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b87954-ead9-4338-b6cd-8f3a03def3af admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:21.968028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b12559-970d-467a-818b-27b8d11d3130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:21.968763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b12559-970d-467a-818b-27b8d11d3130 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:21.969043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b12559-970d-467a-818b-27b8d11d3130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:21.969319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b12559-970d-467a-818b-27b8d11d3130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:21.970092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b12559-970d-467a-818b-27b8d11d3130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:21.971727 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b12559-970d-467a-818b-27b8d11d3130 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:21.983101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 945/1876] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:21.987436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce35a74-3547-4179-9a40-6090bba2bdb3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:21.998428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce35a74-3547-4179-9a40-6090bba2bdb3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:22.003462 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20378391-b10e-449f-825f-ec3f48876238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:22.004374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20378391-b10e-449f-825f-ec3f48876238 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:22.004662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20378391-b10e-449f-825f-ec3f48876238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:22.004973 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20378391-b10e-449f-825f-ec3f48876238 admin admin] PATH_INFO: `/v3/projects/830ca68f5557489d8d77070e38f31829/users/9bf29bc77cab4d82bc37e1ab60a2a621/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:22.021034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20378391-b10e-449f-825f-ec3f48876238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830ca68f5557489d8d77070e38f31829, user_id=9bf29bc77cab4d82bc37e1ab60a2a621, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:22.023838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20378391-b10e-449f-825f-ec3f48876238 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:22.038731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 932/1877] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:21 2022] PUT /identity/v3/projects/830ca68f5557489d8d77070e38f31829/users/9bf29bc77cab4d82bc37e1ab60a2a621/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:22.049409 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee754de-87a4-47ac-b99f-6057e14150f4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:22.049710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee754de-87a4-47ac-b99f-6057e14150f4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:22.050017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee754de-87a4-47ac-b99f-6057e14150f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:22.088273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-aee754de-87a4-47ac-b99f-6057e14150f4 None None] MFA Rules not processed for user `9bf29bc77cab4d82bc37e1ab60a2a621`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:22.090064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-aee754de-87a4-47ac-b99f-6057e14150f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:22.174884 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 946/1878] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:22.182779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20378391-b10e-449f-825f-ec3f48876238 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:22.184634 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-20378391-b10e-449f-825f-ec3f48876238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:22.258093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20378391-b10e-449f-825f-ec3f48876238 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:22.263104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770fc74-b0da-47bd-a2a7-4373a1906869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:22.264026 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770fc74-b0da-47bd-a2a7-4373a1906869 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:22.264411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770fc74-b0da-47bd-a2a7-4373a1906869 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:22.264769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770fc74-b0da-47bd-a2a7-4373a1906869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:22.276582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770fc74-b0da-47bd-a2a7-4373a1906869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:22.278535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770fc74-b0da-47bd-a2a7-4373a1906869 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:22.289811 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 933/1879] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:32.166223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:32.285075 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=2,keystone:SELECT=56,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:33.468700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee754de-87a4-47ac-b99f-6057e14150f4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:33.552461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee754de-87a4-47ac-b99f-6057e14150f4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:33.557918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:33.558952 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:33.559304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:33.559666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] PATH_INFO: `/v3/users/9bf29bc77cab4d82bc37e1ab60a2a621` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:33.560997 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf29bc77cab4d82bc37e1ab60a2a621)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:33.564352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:33.623619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf29bc77cab4d82bc37e1ab60a2a621'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:33.624335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf29bc77cab4d82bc37e1ab60a2a621'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:33.635624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf29bc77cab4d82bc37e1ab60a2a621'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:33.644360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf29bc77cab4d82bc37e1ab60a2a621'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:33.645013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf29bc77cab4d82bc37e1ab60a2a621'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:33.664680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 947/1880] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:33 2022] DELETE /identity/v3/users/9bf29bc77cab4d82bc37e1ab60a2a621 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:33.669002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f37a5e-e1d0-4918-bccd-c247b3950dc1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:33.669523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f37a5e-e1d0-4918-bccd-c247b3950dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:33.669631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f37a5e-e1d0-4918-bccd-c247b3950dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:33.688872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a8f37a5e-e1d0-4918-bccd-c247b3950dc1 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:33.690121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a8f37a5e-e1d0-4918-bccd-c247b3950dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:33.749225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:33.750728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:33.779190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 934/1881] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:33.790306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f37a5e-e1d0-4918-bccd-c247b3950dc1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:33.791711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a8f37a5e-e1d0-4918-bccd-c247b3950dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:33.824690 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fbd22-21dc-435e-8cc2-8329a9b5d731 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:33.829359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26cef1-19a7-4ea2-893b-a5b5e3378cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:33.830539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26cef1-19a7-4ea2-893b-a5b5e3378cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:33.830985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26cef1-19a7-4ea2-893b-a5b5e3378cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:33.831398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26cef1-19a7-4ea2-893b-a5b5e3378cbe admin admin] PATH_INFO: `/v3/users/c809c1a6bf3f41948a0b9171f30ac1e5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:33.832921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26cef1-19a7-4ea2-893b-a5b5e3378cbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c809c1a6bf3f41948a0b9171f30ac1e5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:33.834582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26cef1-19a7-4ea2-893b-a5b5e3378cbe admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:33.856124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f37a5e-e1d0-4918-bccd-c247b3950dc1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:33.859776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:33.860388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b878d0-35a3-4941-b8eb-2e4b290d8c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:33.861081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b878d0-35a3-4941-b8eb-2e4b290d8c4c service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:33.861499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b878d0-35a3-4941-b8eb-2e4b290d8c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:33.861781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b878d0-35a3-4941-b8eb-2e4b290d8c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:33.873027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b878d0-35a3-4941-b8eb-2e4b290d8c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:33.874816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b878d0-35a3-4941-b8eb-2e4b290d8c4c service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:33.887021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 935/1882] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:33.887844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b878d0-35a3-4941-b8eb-2e4b290d8c4c service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:33.889109 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b0b878d0-35a3-4941-b8eb-2e4b290d8c4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:33.893569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7e26cef1-19a7-4ea2-893b-a5b5e3378cbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c809c1a6bf3f41948a0b9171f30ac1e5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:33.894461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7e26cef1-19a7-4ea2-893b-a5b5e3378cbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c809c1a6bf3f41948a0b9171f30ac1e5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:33.903538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b878d0-35a3-4941-b8eb-2e4b290d8c4c service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:33.905696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7e26cef1-19a7-4ea2-893b-a5b5e3378cbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c809c1a6bf3f41948a0b9171f30ac1e5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:33.909104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39241bc-86d7-44aa-a16d-5114fab1e110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:33.910828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39241bc-86d7-44aa-a16d-5114fab1e110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:33.911118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39241bc-86d7-44aa-a16d-5114fab1e110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:33.911447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39241bc-86d7-44aa-a16d-5114fab1e110 admin admin] PATH_INFO: `/v3/users/6877b13897804aa48c58bed4f7f80b81` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:33.912661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39241bc-86d7-44aa-a16d-5114fab1e110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6877b13897804aa48c58bed4f7f80b81)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:33.914713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39241bc-86d7-44aa-a16d-5114fab1e110 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:33.915422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7e26cef1-19a7-4ea2-893b-a5b5e3378cbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c809c1a6bf3f41948a0b9171f30ac1e5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:33.916155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7e26cef1-19a7-4ea2-893b-a5b5e3378cbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c809c1a6bf3f41948a0b9171f30ac1e5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:33.945800 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 948/1883] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:33 2022] DELETE /identity/v3/users/c809c1a6bf3f41948a0b9171f30ac1e5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:33.950481 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7accd-08e2-422a-93a0-2913f4c45d30 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:33.950856 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7accd-08e2-422a-93a0-2913f4c45d30 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:33.951149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7accd-08e2-422a-93a0-2913f4c45d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:33.993407 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e6a7accd-08e2-422a-93a0-2913f4c45d30 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:33.994745 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e6a7accd-08e2-422a-93a0-2913f4c45d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:34.009555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f39241bc-86d7-44aa-a16d-5114fab1e110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6877b13897804aa48c58bed4f7f80b81'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:34.012196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f39241bc-86d7-44aa-a16d-5114fab1e110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6877b13897804aa48c58bed4f7f80b81'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:34.013849 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f39241bc-86d7-44aa-a16d-5114fab1e110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6877b13897804aa48c58bed4f7f80b81'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:34.035223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f39241bc-86d7-44aa-a16d-5114fab1e110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6877b13897804aa48c58bed4f7f80b81'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:34.047202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f39241bc-86d7-44aa-a16d-5114fab1e110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6877b13897804aa48c58bed4f7f80b81'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:34.073876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 936/1884] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:33 2022] DELETE /identity/v3/users/6877b13897804aa48c58bed4f7f80b81 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:34.080260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6beef-5a0b-431d-b64d-56869588042b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:34.080910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6beef-5a0b-431d-b64d-56869588042b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:34.081374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6beef-5a0b-431d-b64d-56869588042b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:34.096101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 949/1885] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:15:34.104790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-84f6beef-5a0b-431d-b64d-56869588042b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:34.106468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7accd-08e2-422a-93a0-2913f4c45d30 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:34.106593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-84f6beef-5a0b-431d-b64d-56869588042b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:34.107939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e6a7accd-08e2-422a-93a0-2913f4c45d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:34.130979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:34.203579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 937/1886] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:34.224547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7accd-08e2-422a-93a0-2913f4c45d30 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:34.269189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6beef-5a0b-431d-b64d-56869588042b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:34.271342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-84f6beef-5a0b-431d-b64d-56869588042b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:34.322820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6057ced-f494-4022-a16d-a1549e0d75db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:34.324212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6057ced-f494-4022-a16d-a1549e0d75db service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:34.324212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6057ced-f494-4022-a16d-a1549e0d75db service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:34.324684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6057ced-f494-4022-a16d-a1549e0d75db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:34.339011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6057ced-f494-4022-a16d-a1549e0d75db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:34.341293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6057ced-f494-4022-a16d-a1549e0d75db service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:34.348691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:34.364158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 950/1887] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:34.433367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6beef-5a0b-431d-b64d-56869588042b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:34.460026 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e8d1b-8cc3-46f1-b5b0-f7f6ff0d2f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:34.469668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e8d1b-8cc3-46f1-b5b0-f7f6ff0d2f57 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:34.469996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e8d1b-8cc3-46f1-b5b0-f7f6ff0d2f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:34.470280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e8d1b-8cc3-46f1-b5b0-f7f6ff0d2f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:34.491759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e8d1b-8cc3-46f1-b5b0-f7f6ff0d2f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:34.493996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e8d1b-8cc3-46f1-b5b0-f7f6ff0d2f57 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:34.519852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 938/1888] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:34.919334 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6057ced-f494-4022-a16d-a1549e0d75db service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:34.921344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c6057ced-f494-4022-a16d-a1549e0d75db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:34.952769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6057ced-f494-4022-a16d-a1549e0d75db service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:34.974372 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:34.976469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:34.976592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:34.977225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] PATH_INFO: `/v3/projects/20ce552af8eb4077b9971db6d235ed65` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:34.979411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=20ce552af8eb4077b9971db6d235ed65)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:34.984613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:35.092556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20ce552af8eb4077b9971db6d235ed65', 'request_id': 'req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.103871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20ce552af8eb4077b9971db6d235ed65', 'request_id': 'req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.115347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20ce552af8eb4077b9971db6d235ed65', 'request_id': 'req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.135774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] The token cache is being invalidate because project 20ce552af8eb4077b9971db6d235ed65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:35.136296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.139411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.142135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 951/1889] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:34 2022] DELETE /identity/v3/projects/20ce552af8eb4077b9971db6d235ed65 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:35.147870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e8d1b-8cc3-46f1-b5b0-f7f6ff0d2f57 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:35.167902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e8d1b-8cc3-46f1-b5b0-f7f6ff0d2f57 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:35.176811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:35.179235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:35.252586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:35.255004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:35.255408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:35.256009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] PATH_INFO: `/v3/users/056f86dc774e451ca3d140e2cb708d3a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:35.258411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=056f86dc774e451ca3d140e2cb708d3a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:35.263040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:35.321585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0bf69-b6bb-4e5e-a55b-b986f5028eef admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:35.338511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:35.339483 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:35.339863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:35.340088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] PATH_INFO: `/v3/projects/830ca68f5557489d8d77070e38f31829` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:35.342378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=830ca68f5557489d8d77070e38f31829)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:35.345134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:35.375097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '056f86dc774e451ca3d140e2cb708d3a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.375883 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '056f86dc774e451ca3d140e2cb708d3a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.378432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '056f86dc774e451ca3d140e2cb708d3a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.389514 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '056f86dc774e451ca3d140e2cb708d3a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.396661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:35.399048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '056f86dc774e451ca3d140e2cb708d3a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.410896 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830ca68f5557489d8d77070e38f31829', 'request_id': 'req-a1fcdbad-ff6d-4eb8-933b-c6866a475674'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.423763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830ca68f5557489d8d77070e38f31829', 'request_id': 'req-a1fcdbad-ff6d-4eb8-933b-c6866a475674'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.432583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830ca68f5557489d8d77070e38f31829', 'request_id': 'req-a1fcdbad-ff6d-4eb8-933b-c6866a475674'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.435294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 939/1890] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:35 2022] DELETE /identity/v3/users/056f86dc774e451ca3d140e2cb708d3a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:35.436171 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:35.438217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:35.465209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] The token cache is being invalidate because project 830ca68f5557489d8d77070e38f31829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:35.465672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.466575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a1fcdbad-ff6d-4eb8-933b-c6866a475674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.468506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 952/1891] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:35 2022] DELETE /identity/v3/projects/830ca68f5557489d8d77070e38f31829 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:35.482216 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba29e92-0bfc-4757-aaf7-e1cf52f8bb6e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:35.482502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba29e92-0bfc-4757-aaf7-e1cf52f8bb6e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:35.482777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba29e92-0bfc-4757-aaf7-e1cf52f8bb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:35.525959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4ba29e92-0bfc-4757-aaf7-e1cf52f8bb6e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:35.527855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4ba29e92-0bfc-4757-aaf7-e1cf52f8bb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:35.538629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:35.565809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8964c-3b56-40ac-b8e4-e34ce007fd0e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:35.583021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:35.584551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:35.584721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:35.585719 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] PATH_INFO: `/v3/projects/4b9804d46bef4df5a33201156023dbb3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:35.587495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b9804d46bef4df5a33201156023dbb3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:35.599509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:35.648330 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 953/1892] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:35.657701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f399f-4940-4dde-a1c3-10b8e31fd69a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:35.658375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f399f-4940-4dde-a1c3-10b8e31fd69a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:35.658972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f399f-4940-4dde-a1c3-10b8e31fd69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:35.677123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b9804d46bef4df5a33201156023dbb3', 'request_id': 'req-eecb5248-d493-4f69-84bd-c732dd23d289'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.684679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f60f399f-4940-4dde-a1c3-10b8e31fd69a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:35.685985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f60f399f-4940-4dde-a1c3-10b8e31fd69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:35.698426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b9804d46bef4df5a33201156023dbb3', 'request_id': 'req-eecb5248-d493-4f69-84bd-c732dd23d289'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.706584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b9804d46bef4df5a33201156023dbb3', 'request_id': 'req-eecb5248-d493-4f69-84bd-c732dd23d289'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.717493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] The token cache is being invalidate because project 4b9804d46bef4df5a33201156023dbb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:35.717825 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.718929 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.720809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 940/1893] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:35 2022] DELETE /identity/v3/projects/4b9804d46bef4df5a33201156023dbb3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:35.724403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:35.725816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:35.767799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 954/1894] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:35.767799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f399f-4940-4dde-a1c3-10b8e31fd69a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:35.767799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f60f399f-4940-4dde-a1c3-10b8e31fd69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:35.792492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f399f-4940-4dde-a1c3-10b8e31fd69a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:35.798257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:35.799680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:35.799975 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:35.800302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] PATH_INFO: `/v3/users/38c0322c2d0c4d5db70d19fd159de037` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:35.801564 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c0322c2d0c4d5db70d19fd159de037)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:35.803899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:35.817068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb5248-d493-4f69-84bd-c732dd23d289 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:35.827391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89667a-a804-4123-bca4-f2176260af5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:35.828292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89667a-a804-4123-bca4-f2176260af5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:35.828575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89667a-a804-4123-bca4-f2176260af5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:35.828851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89667a-a804-4123-bca4-f2176260af5e admin admin] PATH_INFO: `/v3/users/e0c6f5504b304d6f86794e0110880a5c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:35.839218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89667a-a804-4123-bca4-f2176260af5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0c6f5504b304d6f86794e0110880a5c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:35.841330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89667a-a804-4123-bca4-f2176260af5e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:35.857784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:35.892124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c0322c2d0c4d5db70d19fd159de037'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.893331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c0322c2d0c4d5db70d19fd159de037'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.910366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c0322c2d0c4d5db70d19fd159de037'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.918143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ef89667a-a804-4123-bca4-f2176260af5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0c6f5504b304d6f86794e0110880a5c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.918859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ef89667a-a804-4123-bca4-f2176260af5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0c6f5504b304d6f86794e0110880a5c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.920245 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ef89667a-a804-4123-bca4-f2176260af5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0c6f5504b304d6f86794e0110880a5c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.926401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c0322c2d0c4d5db70d19fd159de037'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.927742 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c0322c2d0c4d5db70d19fd159de037'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.931136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ef89667a-a804-4123-bca4-f2176260af5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0c6f5504b304d6f86794e0110880a5c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.948006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ef89667a-a804-4123-bca4-f2176260af5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0c6f5504b304d6f86794e0110880a5c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:35.950116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 955/1895] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:35 2022] DELETE /identity/v3/users/38c0322c2d0c4d5db70d19fd159de037 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:35.951032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:35.952426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:36.000878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 941/1896] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:35 2022] DELETE /identity/v3/users/e0c6f5504b304d6f86794e0110880a5c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:36.003996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:36.005732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a861ef1-1e3a-4282-a404-1d0d0b52bfd3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.006055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a861ef1-1e3a-4282-a404-1d0d0b52bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.006338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a861ef1-1e3a-4282-a404-1d0d0b52bfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.056789 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4a861ef1-1e3a-4282-a404-1d0d0b52bfd3 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:36.058077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4a861ef1-1e3a-4282-a404-1d0d0b52bfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:36.085110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e31b1b-3188-43d9-953c-7d49cea107bd admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:36.134009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 942/1897] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:36.141991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a861ef1-1e3a-4282-a404-1d0d0b52bfd3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:36.145253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4a861ef1-1e3a-4282-a404-1d0d0b52bfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:36.207674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37653818-1afa-4e2d-bba3-437a7afd8cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:36.209831 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37653818-1afa-4e2d-bba3-437a7afd8cfd admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.210219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37653818-1afa-4e2d-bba3-437a7afd8cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.210572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37653818-1afa-4e2d-bba3-437a7afd8cfd admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.213142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37653818-1afa-4e2d-bba3-437a7afd8cfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:36.218982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37653818-1afa-4e2d-bba3-437a7afd8cfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:36.221348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37653818-1afa-4e2d-bba3-437a7afd8cfd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:36.241645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a861ef1-1e3a-4282-a404-1d0d0b52bfd3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:36.247408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c39bf1-35fd-4899-b4ed-2fe572a1b18a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:36.248382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c39bf1-35fd-4899-b4ed-2fe572a1b18a service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.248725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c39bf1-35fd-4899-b4ed-2fe572a1b18a service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.249012 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c39bf1-35fd-4899-b4ed-2fe572a1b18a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.249400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 956/1898] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:36.250299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37653818-1afa-4e2d-bba3-437a7afd8cfd admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:36.256155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-37653818-1afa-4e2d-bba3-437a7afd8cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:36.263001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c39bf1-35fd-4899-b4ed-2fe572a1b18a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:36.263905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c39bf1-35fd-4899-b4ed-2fe572a1b18a service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:36.277481 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 943/1899] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:15:36.284102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37653818-1afa-4e2d-bba3-437a7afd8cfd admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:36.314699 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404c8f3-42fe-4161-8a43-2a75f2b3f553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:36.315577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404c8f3-42fe-4161-8a43-2a75f2b3f553 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.316017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404c8f3-42fe-4161-8a43-2a75f2b3f553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.316396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404c8f3-42fe-4161-8a43-2a75f2b3f553 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.317600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404c8f3-42fe-4161-8a43-2a75f2b3f553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:36.322870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404c8f3-42fe-4161-8a43-2a75f2b3f553 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:36.333370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c39bf1-35fd-4899-b4ed-2fe572a1b18a service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:36.335023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-01c39bf1-35fd-4899-b4ed-2fe572a1b18a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:36.357805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c39bf1-35fd-4899-b4ed-2fe572a1b18a service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:36.367273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:36.368312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.368623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.368871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] PATH_INFO: `/v3/projects/33e5c18fc2164ce1b2ec6ab10781a99c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.370759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e5c18fc2164ce1b2ec6ab10781a99c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:36.382200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:36.395190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 957/1900] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:36 2022] POST /identity/v3/projects => generated 369 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:36.396525 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404c8f3-42fe-4161-8a43-2a75f2b3f553 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:36.467887 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:36.499117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e5c18fc2164ce1b2ec6ab10781a99c', 'request_id': 'req-8a51a96b-240b-4782-b01a-b9a32b40014c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:36.510815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e5c18fc2164ce1b2ec6ab10781a99c', 'request_id': 'req-8a51a96b-240b-4782-b01a-b9a32b40014c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:36.511447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404c8f3-42fe-4161-8a43-2a75f2b3f553 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:36.519756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e5c18fc2164ce1b2ec6ab10781a99c', 'request_id': 'req-8a51a96b-240b-4782-b01a-b9a32b40014c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:36.534928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] The token cache is being invalidate because project 33e5c18fc2164ce1b2ec6ab10781a99c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:36.535225 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:36.536296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8a51a96b-240b-4782-b01a-b9a32b40014c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:36.538277 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 944/1901] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:36 2022] DELETE /identity/v3/projects/33e5c18fc2164ce1b2ec6ab10781a99c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:36.545982 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95446e1a-ff3f-49ae-9c92-7de04a09e070 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.546316 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95446e1a-ff3f-49ae-9c92-7de04a09e070 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.546582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95446e1a-ff3f-49ae-9c92-7de04a09e070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.574878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-95446e1a-ff3f-49ae-9c92-7de04a09e070 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:36.577113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-95446e1a-ff3f-49ae-9c92-7de04a09e070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:36.631492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:36.657503 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e613b36-de0d-4dfd-be39-af863e85aeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:36.658809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e613b36-de0d-4dfd-be39-af863e85aeaf admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.659422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e613b36-de0d-4dfd-be39-af863e85aeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.659422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e613b36-de0d-4dfd-be39-af863e85aeaf admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.660172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e613b36-de0d-4dfd-be39-af863e85aeaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:36.663146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e613b36-de0d-4dfd-be39-af863e85aeaf admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:36.664260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 945/1902] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:36.672380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621a4a7-649e-4782-abcb-7f19b4929496 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.672689 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621a4a7-649e-4782-abcb-7f19b4929496 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.672943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621a4a7-649e-4782-abcb-7f19b4929496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.702679 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c621a4a7-649e-4782-abcb-7f19b4929496 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:36.704053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 958/1903] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:36 2022] POST /identity/v3/users => generated 285 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:36.704166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c621a4a7-649e-4782-abcb-7f19b4929496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:36.708341 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e613b36-de0d-4dfd-be39-af863e85aeaf admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:36.712009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9e613b36-de0d-4dfd-be39-af863e85aeaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:36.731306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 946/1904] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:15:36.736236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621a4a7-649e-4782-abcb-7f19b4929496 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:36.745412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e613b36-de0d-4dfd-be39-af863e85aeaf admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:36.759690 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de8b31-d0a5-4d74-8e37-f9b585944ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:36.760583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de8b31-d0a5-4d74-8e37-f9b585944ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.761065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de8b31-d0a5-4d74-8e37-f9b585944ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.761526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de8b31-d0a5-4d74-8e37-f9b585944ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.762485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de8b31-d0a5-4d74-8e37-f9b585944ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:36.764176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de8b31-d0a5-4d74-8e37-f9b585944ef4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:36.774208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621a4a7-649e-4782-abcb-7f19b4929496 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:36.787049 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 959/1905] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:36.787253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cb4ce-0dfa-421e-9fca-156ac30823e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:36.787354 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de8b31-d0a5-4d74-8e37-f9b585944ef4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:36.787448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cb4ce-0dfa-421e-9fca-156ac30823e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.787448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cb4ce-0dfa-421e-9fca-156ac30823e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.787448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cb4ce-0dfa-421e-9fca-156ac30823e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.787448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cb4ce-0dfa-421e-9fca-156ac30823e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:36.787694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cb4ce-0dfa-421e-9fca-156ac30823e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:36.789669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-95de8b31-d0a5-4d74-8e37-f9b585944ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:36.790390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cb4ce-0dfa-421e-9fca-156ac30823e4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:36.805075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de8b31-d0a5-4d74-8e37-f9b585944ef4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:36.816653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:36.817345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 947/1906] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:36.821575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cb4ce-0dfa-421e-9fca-156ac30823e4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:36.830844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.831902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.833012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] PATH_INFO: `/v3/projects/107bbbb5785047dc85b7aa5bdf532c85` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.833147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cb4ce-0dfa-421e-9fca-156ac30823e4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:36.837871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=107bbbb5785047dc85b7aa5bdf532c85)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:36.839167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28ddc0-7049-4701-b115-755820a753d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:36.840171 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:36.841516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28ddc0-7049-4701-b115-755820a753d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.841867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28ddc0-7049-4701-b115-755820a753d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.842303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28ddc0-7049-4701-b115-755820a753d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.843132 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28ddc0-7049-4701-b115-755820a753d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:36.844814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28ddc0-7049-4701-b115-755820a753d5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:36.860396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 948/1907] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:36.862275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c745d9-ffe5-4f11-9c04-04af630bf838 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.862275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c745d9-ffe5-4f11-9c04-04af630bf838 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.862275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c745d9-ffe5-4f11-9c04-04af630bf838 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.864250 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 949/1908] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:15:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:15:36.865195 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c745d9-ffe5-4f11-9c04-04af630bf838 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:36.867121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-62c745d9-ffe5-4f11-9c04-04af630bf838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:36.889262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c745d9-ffe5-4f11-9c04-04af630bf838 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:36.905329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:36.906732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:36.910958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:36.912463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] PATH_INFO: `/v3/projects/44af3132624249729b6150d85a707f23` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:36.916400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44af3132624249729b6150d85a707f23)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:36.919190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:36.925424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '107bbbb5785047dc85b7aa5bdf532c85', 'request_id': 'req-390e51b7-48f4-420c-b62a-ceee05173a44'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:36.938916 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '107bbbb5785047dc85b7aa5bdf532c85', 'request_id': 'req-390e51b7-48f4-420c-b62a-ceee05173a44'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:36.952293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '107bbbb5785047dc85b7aa5bdf532c85', 'request_id': 'req-390e51b7-48f4-420c-b62a-ceee05173a44'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:36.970768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] The token cache is being invalidate because project 107bbbb5785047dc85b7aa5bdf532c85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:36.971370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:36.972798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:36.976213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 960/1909] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:36 2022] DELETE /identity/v3/projects/107bbbb5785047dc85b7aa5bdf532c85 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:36.977168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:36.986727 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:36.990209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44af3132624249729b6150d85a707f23', 'request_id': 'req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.006787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44af3132624249729b6150d85a707f23', 'request_id': 'req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.023453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44af3132624249729b6150d85a707f23', 'request_id': 'req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.041110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] The token cache is being invalidate because project 44af3132624249729b6150d85a707f23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:37.041441 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.042166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.044453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 950/1910] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:36 2022] DELETE /identity/v3/projects/44af3132624249729b6150d85a707f23 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:37.045068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:37.046973 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.058464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:37.091455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e51b7-48f4-420c-b62a-ceee05173a44 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:37.170603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede31c9-37cd-4c94-9a1e-275be771d3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:37.171290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede31c9-37cd-4c94-9a1e-275be771d3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.171576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede31c9-37cd-4c94-9a1e-275be771d3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.171856 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede31c9-37cd-4c94-9a1e-275be771d3a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.172607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede31c9-37cd-4c94-9a1e-275be771d3a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:37.173396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf88ab-c8e4-4c29-a131-8ab7307a2085 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:37.174983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede31c9-37cd-4c94-9a1e-275be771d3a3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:37.178794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:37.178951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e60c1-f486-45c9-8799-9f3d80d196bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:37.179849 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e60c1-f486-45c9-8799-9f3d80d196bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.180084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e60c1-f486-45c9-8799-9f3d80d196bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.180360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e60c1-f486-45c9-8799-9f3d80d196bf admin admin] PATH_INFO: `/v3/projects/fbaa9b9c6e0b4180bb267874bbbb05ad/users/03ec9afc1d8f47f69e0b9656dbd4725c/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.197945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e60c1-f486-45c9-8799-9f3d80d196bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbaa9b9c6e0b4180bb267874bbbb05ad, user_id=03ec9afc1d8f47f69e0b9656dbd4725c, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:37.200938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e60c1-f486-45c9-8799-9f3d80d196bf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:37.210426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 961/1911] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:36 2022] POST /identity/v3/projects => generated 391 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:37.210426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9f92c-38da-4ca2-bb50-701641d843af None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.210426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9f92c-38da-4ca2-bb50-701641d843af None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.210426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9f92c-38da-4ca2-bb50-701641d843af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.220254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 951/1912] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:37 2022] PUT /identity/v3/projects/fbaa9b9c6e0b4180bb267874bbbb05ad/users/03ec9afc1d8f47f69e0b9656dbd4725c/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:37.221541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e60c1-f486-45c9-8799-9f3d80d196bf admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:37.230080 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2d9e60c1-f486-45c9-8799-9f3d80d196bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.232146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2ec9f92c-38da-4ca2-bb50-701641d843af None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:37.233413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2ec9f92c-38da-4ca2-bb50-701641d843af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.304091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e60c1-f486-45c9-8799-9f3d80d196bf admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:37.309156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:37.310603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.310893 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.311174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] PATH_INFO: `/v3/users/0a9f35fa120e4de69364c76fab8d7827` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.312391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a9f35fa120e4de69364c76fab8d7827)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:37.314420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:37.318616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 962/1913] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:15:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:37.319579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9f92c-38da-4ca2-bb50-701641d843af None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:37.321009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2ec9f92c-38da-4ca2-bb50-701641d843af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.334854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9f92c-38da-4ca2-bb50-701641d843af None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:37.339657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4cf94-06bf-4292-9044-e46cfdf9b2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:37.340595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4cf94-06bf-4292-9044-e46cfdf9b2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.340937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4cf94-06bf-4292-9044-e46cfdf9b2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.341255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4cf94-06bf-4292-9044-e46cfdf9b2c9 admin admin] PATH_INFO: `/v3/users/997657bd8b8646e19defff8bd43bbde1` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.342618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f4cf94-06bf-4292-9044-e46cfdf9b2c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=997657bd8b8646e19defff8bd43bbde1)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:37.344867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f4cf94-06bf-4292-9044-e46cfdf9b2c9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:37.367737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a9f35fa120e4de69364c76fab8d7827'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.368554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a9f35fa120e4de69364c76fab8d7827'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.369277 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a9f35fa120e4de69364c76fab8d7827'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.379345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a9f35fa120e4de69364c76fab8d7827'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.390614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a9f35fa120e4de69364c76fab8d7827'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.408710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-57f4cf94-06bf-4292-9044-e46cfdf9b2c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '997657bd8b8646e19defff8bd43bbde1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.409658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-57f4cf94-06bf-4292-9044-e46cfdf9b2c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '997657bd8b8646e19defff8bd43bbde1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.420518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 952/1914] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:37 2022] DELETE /identity/v3/users/0a9f35fa120e4de69364c76fab8d7827 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:37.421372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:37.422990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.424537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-57f4cf94-06bf-4292-9044-e46cfdf9b2c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '997657bd8b8646e19defff8bd43bbde1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.432744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-57f4cf94-06bf-4292-9044-e46cfdf9b2c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '997657bd8b8646e19defff8bd43bbde1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.433848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-57f4cf94-06bf-4292-9044-e46cfdf9b2c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '997657bd8b8646e19defff8bd43bbde1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.436315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:37.460807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 963/1915] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:37 2022] DELETE /identity/v3/users/997657bd8b8646e19defff8bd43bbde1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:37.462368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc69b4-1ae7-4f7e-b437-a78fcead0bde None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.462758 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc69b4-1ae7-4f7e-b437-a78fcead0bde None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.463047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc69b4-1ae7-4f7e-b437-a78fcead0bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.499163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-71bc69b4-1ae7-4f7e-b437-a78fcead0bde None None] MFA Rules not processed for user `03ec9afc1d8f47f69e0b9656dbd4725c`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:37.500579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-71bc69b4-1ae7-4f7e-b437-a78fcead0bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.517017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263ea7e-0db8-4806-80a1-7d349bb44cb6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:37.571361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:37.621774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 964/1916] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:37.621774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc69b4-1ae7-4f7e-b437-a78fcead0bde None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:37.621774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-71bc69b4-1ae7-4f7e-b437-a78fcead0bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.623902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41cafa-56e0-4e35-849f-d922914a8b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:37.624767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41cafa-56e0-4e35-849f-d922914a8b1c admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.625063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41cafa-56e0-4e35-849f-d922914a8b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.625347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41cafa-56e0-4e35-849f-d922914a8b1c admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.626088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41cafa-56e0-4e35-849f-d922914a8b1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:37.628519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41cafa-56e0-4e35-849f-d922914a8b1c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:37.657924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 953/1917] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:37 2022] POST /identity/v3/users => generated 296 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:37.657924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41cafa-56e0-4e35-849f-d922914a8b1c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:37.660520 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5c41cafa-56e0-4e35-849f-d922914a8b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.673474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41cafa-56e0-4e35-849f-d922914a8b1c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:37.679864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:37.681476 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.681760 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.682138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] PATH_INFO: `/v3/users/6e729089f66946bb97fbfc0faf45463c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.683448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e729089f66946bb97fbfc0faf45463c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:37.685932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:37.732314 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc69b4-1ae7-4f7e-b437-a78fcead0bde None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:37.739155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f2e59-2d47-41b3-a1a1-bfd1cfb03660 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:37.740041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2e59-2d47-41b3-a1a1-bfd1cfb03660 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.740438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2e59-2d47-41b3-a1a1-bfd1cfb03660 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.740806 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2e59-2d47-41b3-a1a1-bfd1cfb03660 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.742930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-316f2e59-2d47-41b3-a1a1-bfd1cfb03660 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.749466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e729089f66946bb97fbfc0faf45463c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.750740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e729089f66946bb97fbfc0faf45463c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.751401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e729089f66946bb97fbfc0faf45463c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.757965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f2e59-2d47-41b3-a1a1-bfd1cfb03660 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:37.760647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f2e59-2d47-41b3-a1a1-bfd1cfb03660 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:37.763691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e729089f66946bb97fbfc0faf45463c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.774303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 965/1918] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:37.775473 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f2e59-2d47-41b3-a1a1-bfd1cfb03660 service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:37.775902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e729089f66946bb97fbfc0faf45463c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:37.777819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-316f2e59-2d47-41b3-a1a1-bfd1cfb03660 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.801344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 954/1919] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:37 2022] DELETE /identity/v3/users/6e729089f66946bb97fbfc0faf45463c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:37.802273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:37.804473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.850126 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:37.888456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f2e59-2d47-41b3-a1a1-bfd1cfb03660 service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:37.912844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467ca4d-b0c9-4450-b8d2-4bf44048890b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:37.918146 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0ace7-73a8-40fb-bcc8-a90ca1cc3e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:37.918901 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0ace7-73a8-40fb-bcc8-a90ca1cc3e06 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.919231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0ace7-73a8-40fb-bcc8-a90ca1cc3e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.919706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0ace7-73a8-40fb-bcc8-a90ca1cc3e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.920427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0ace7-73a8-40fb-bcc8-a90ca1cc3e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:37.922462 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0ace7-73a8-40fb-bcc8-a90ca1cc3e06 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:37.936919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 955/1920] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:37.938519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7d024-833e-4dbd-9151-ff40dc39e161 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.938878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7d024-833e-4dbd-9151-ff40dc39e161 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.939244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7d024-833e-4dbd-9151-ff40dc39e161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.956770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1231759-789a-4b91-80b4-7aa6319efd74 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a90e384504e8d80264d6bf20f1d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:37.957441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1231759-789a-4b91-80b4-7aa6319efd74 service glance] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.957680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8fa7d024-833e-4dbd-9151-ff40dc39e161 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:37.957805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1231759-789a-4b91-80b4-7aa6319efd74 service glance] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.958058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1231759-789a-4b91-80b4-7aa6319efd74 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.958961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8fa7d024-833e-4dbd-9151-ff40dc39e161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.959839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d1231759-789a-4b91-80b4-7aa6319efd74 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.967829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 956/1921] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:37.968644 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7d024-833e-4dbd-9151-ff40dc39e161 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:37.971920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8fa7d024-833e-4dbd-9151-ff40dc39e161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:37.987230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7d024-833e-4dbd-9151-ff40dc39e161 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:37.992227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:37.993172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:37.993531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:37.993811 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] PATH_INFO: `/v3/projects/81bc8df29a9e471dae661f680cc54b6a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:37.995043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] RBAC: Authorizing `identity:delete_project(project_id=81bc8df29a9e471dae661f680cc54b6a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:37.996955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:37.998320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:38.026819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1231759-789a-4b91-80b4-7aa6319efd74 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:38.028284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1231759-789a-4b91-80b4-7aa6319efd74 service glance] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:38.038036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 966/1922] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:38.038959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1231759-789a-4b91-80b4-7aa6319efd74 service glance] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:38.040464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d1231759-789a-4b91-80b4-7aa6319efd74 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:38.058862 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81bc8df29a9e471dae661f680cc54b6a', 'request_id': 'req-6af4791d-3c49-431d-8921-a1224db1e70d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:38.059904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1231759-789a-4b91-80b4-7aa6319efd74 service glance] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:38.086872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81bc8df29a9e471dae661f680cc54b6a', 'request_id': 'req-6af4791d-3c49-431d-8921-a1224db1e70d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:38.095088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81bc8df29a9e471dae661f680cc54b6a', 'request_id': 'req-6af4791d-3c49-431d-8921-a1224db1e70d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:38.108522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] The token cache is being invalidate because project 81bc8df29a9e471dae661f680cc54b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:38.109053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:38.122694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:38.143948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 957/1923] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:37 2022] DELETE /identity/v3/projects/81bc8df29a9e471dae661f680cc54b6a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:38.143948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:38.143948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:38.165298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:38.166234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:38.166647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:38.166879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] PATH_INFO: `/v3/projects/f2118e40506844a8bb83c010c520689d` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:38.173127 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2118e40506844a8bb83c010c520689d)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:38.175177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:38.183240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4791d-3c49-431d-8921-a1224db1e70d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:38.201275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ed74c-74cd-4f7b-8ccd-0ca9f7733258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:38.202008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ed74c-74cd-4f7b-8ccd-0ca9f7733258 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:38.202299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ed74c-74cd-4f7b-8ccd-0ca9f7733258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:38.202580 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ed74c-74cd-4f7b-8ccd-0ca9f7733258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:38.203330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ed74c-74cd-4f7b-8ccd-0ca9f7733258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:38.210371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ed74c-74cd-4f7b-8ccd-0ca9f7733258 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:38.238928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 958/1924] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:38.239852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ed74c-74cd-4f7b-8ccd-0ca9f7733258 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:38.246258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-215ed74c-74cd-4f7b-8ccd-0ca9f7733258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:38.294913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2118e40506844a8bb83c010c520689d', 'request_id': 'req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:38.310071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2118e40506844a8bb83c010c520689d', 'request_id': 'req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:38.319853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2118e40506844a8bb83c010c520689d', 'request_id': 'req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:38.338184 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] The token cache is being invalidate because project f2118e40506844a8bb83c010c520689d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:38.338184 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:38.340430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c2f82b89-0f2b-4a4d-b15a-498c134dcbf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:38.350395 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 967/1925] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:38 2022] DELETE /identity/v3/projects/f2118e40506844a8bb83c010c520689d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:38.351891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091a660-94c1-40d3-b609-c83d2d739da8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:38.352177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091a660-94c1-40d3-b609-c83d2d739da8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:38.352923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091a660-94c1-40d3-b609-c83d2d739da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:38.379036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b091a660-94c1-40d3-b609-c83d2d739da8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:38.380440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b091a660-94c1-40d3-b609-c83d2d739da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:38.384388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ed74c-74cd-4f7b-8ccd-0ca9f7733258 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:38.443491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:38.465358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39f1f2-bf24-499f-8678-3087a8bd3c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:38.466055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39f1f2-bf24-499f-8678-3087a8bd3c93 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:38.466338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39f1f2-bf24-499f-8678-3087a8bd3c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:38.466612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39f1f2-bf24-499f-8678-3087a8bd3c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:38.468865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dc39f1f2-bf24-499f-8678-3087a8bd3c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:38.497070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 968/1926] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:38.504342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091a660-94c1-40d3-b609-c83d2d739da8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:38.504342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b091a660-94c1-40d3-b609-c83d2d739da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:38.509810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:38.514423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091a660-94c1-40d3-b609-c83d2d739da8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:38.522464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc270959-5d70-4b1a-99cd-ef8ee02ce2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:38.523615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc270959-5d70-4b1a-99cd-ef8ee02ce2a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:38.524604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc270959-5d70-4b1a-99cd-ef8ee02ce2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:38.525006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc270959-5d70-4b1a-99cd-ef8ee02ce2a2 admin admin] PATH_INFO: `/v3/projects/a1b0065ad771431784ce393d978942aa/users/7c7736d1595e41aa9ef207b777034096/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:38.544289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc270959-5d70-4b1a-99cd-ef8ee02ce2a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b0065ad771431784ce393d978942aa, user_id=7c7736d1595e41aa9ef207b777034096, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:38.547949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc270959-5d70-4b1a-99cd-ef8ee02ce2a2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:38.572623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 969/1927] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:38 2022] PUT /identity/v3/projects/a1b0065ad771431784ce393d978942aa/users/7c7736d1595e41aa9ef207b777034096/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:38.587720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14cde-ece1-42ae-9086-ca2c2ce363ba None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:38.588290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14cde-ece1-42ae-9086-ca2c2ce363ba None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:38.588863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14cde-ece1-42ae-9086-ca2c2ce363ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:38.615683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39f1f2-bf24-499f-8678-3087a8bd3c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:38.617742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39f1f2-bf24-499f-8678-3087a8bd3c93 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:38.637312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-86a14cde-ece1-42ae-9086-ca2c2ce363ba None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:38.639498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-86a14cde-ece1-42ae-9086-ca2c2ce363ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:38.639972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 959/1928] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:38.642196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6996a2-8d25-47bb-a52a-dc9da0b9e713 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:38.642638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6996a2-8d25-47bb-a52a-dc9da0b9e713 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:38.643052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6996a2-8d25-47bb-a52a-dc9da0b9e713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:38.689021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9f6996a2-8d25-47bb-a52a-dc9da0b9e713 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:38.691575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9f6996a2-8d25-47bb-a52a-dc9da0b9e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:38.820089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 960/1929] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:38.820089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90291a2-26c6-4f11-974f-504234906a51 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:38.820089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90291a2-26c6-4f11-974f-504234906a51 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:38.820089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90291a2-26c6-4f11-974f-504234906a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:38.860545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f90291a2-26c6-4f11-974f-504234906a51 None None] MFA Rules not processed for user `7c7736d1595e41aa9ef207b777034096`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:38.862296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f90291a2-26c6-4f11-974f-504234906a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:38.885317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 970/1930] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:38.885317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a14cde-ece1-42ae-9086-ca2c2ce363ba None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:38.904472 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a14cde-ece1-42ae-9086-ca2c2ce363ba None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:38.918204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbca62b-b69f-4e09-90d4-c37a9673fbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:38.919287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbca62b-b69f-4e09-90d4-c37a9673fbee admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:38.920397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbca62b-b69f-4e09-90d4-c37a9673fbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:38.921774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbca62b-b69f-4e09-90d4-c37a9673fbee admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:38.924107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbca62b-b69f-4e09-90d4-c37a9673fbee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:38.926597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbca62b-b69f-4e09-90d4-c37a9673fbee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:38.933633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbca62b-b69f-4e09-90d4-c37a9673fbee admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:38.971126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 971/1931] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:38.971126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df2610-de5b-4c9b-9970-f4de76f4962c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:38.971126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df2610-de5b-4c9b-9970-f4de76f4962c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:38.971126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df2610-de5b-4c9b-9970-f4de76f4962c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.027972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f0df2610-de5b-4c9b-9970-f4de76f4962c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:39.029373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f0df2610-de5b-4c9b-9970-f4de76f4962c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:39.039171 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 972/1932] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:39.040199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df2610-de5b-4c9b-9970-f4de76f4962c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.051202 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df2610-de5b-4c9b-9970-f4de76f4962c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.054297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 961/1933] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:39.055655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90291a2-26c6-4f11-974f-504234906a51 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.057087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a311e6-81e6-470e-800b-4a24c9eb0156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.058074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a311e6-81e6-470e-800b-4a24c9eb0156 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.058432 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a311e6-81e6-470e-800b-4a24c9eb0156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.058716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a311e6-81e6-470e-800b-4a24c9eb0156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.059485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a311e6-81e6-470e-800b-4a24c9eb0156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.061548 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a311e6-81e6-470e-800b-4a24c9eb0156 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.070137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90291a2-26c6-4f11-974f-504234906a51 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.075198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f5d12-a294-4693-b4a2-2061ac26a993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.075937 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f5d12-a294-4693-b4a2-2061ac26a993 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.076228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f5d12-a294-4693-b4a2-2061ac26a993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.076507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f5d12-a294-4693-b4a2-2061ac26a993 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.077303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f5d12-a294-4693-b4a2-2061ac26a993 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:39.077618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f5d12-a294-4693-b4a2-2061ac26a993 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.079332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f5d12-a294-4693-b4a2-2061ac26a993 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.093028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 962/1934] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:39.093916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f5d12-a294-4693-b4a2-2061ac26a993 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.097850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 973/1935] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:39 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:39.099011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a311e6-81e6-470e-800b-4a24c9eb0156 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.101084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:39.108032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f5d12-a294-4693-b4a2-2061ac26a993 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.201132 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156bd36f-0d95-4d79-84ad-d14bd344e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.203271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bd36f-0d95-4d79-84ad-d14bd344e259 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.203271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bd36f-0d95-4d79-84ad-d14bd344e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.203608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bd36f-0d95-4d79-84ad-d14bd344e259 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.204254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156bd36f-0d95-4d79-84ad-d14bd344e259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.206606 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156bd36f-0d95-4d79-84ad-d14bd344e259 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.208861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a311e6-81e6-470e-800b-4a24c9eb0156 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.214789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1a3df-6976-4478-9a90-afaed4e34ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.215790 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a3df-6976-4478-9a90-afaed4e34ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.216182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a3df-6976-4478-9a90-afaed4e34ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.216539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a3df-6976-4478-9a90-afaed4e34ba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.217398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1a3df-6976-4478-9a90-afaed4e34ba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.219439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1a3df-6976-4478-9a90-afaed4e34ba9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.242765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 963/1936] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:39 2022] POST /identity/v3/projects => generated 391 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:39.243638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156bd36f-0d95-4d79-84ad-d14bd344e259 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.256256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 974/1937] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:39 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:39.256256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1a3df-6976-4478-9a90-afaed4e34ba9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.256256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-dfb1a3df-6976-4478-9a90-afaed4e34ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:39.285197 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:39.317338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156bd36f-0d95-4d79-84ad-d14bd344e259 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.322158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb359ef3-0499-49e4-ae81-ea3de86a8fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.322836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb359ef3-0499-49e4-ae81-ea3de86a8fbb admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.323118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb359ef3-0499-49e4-ae81-ea3de86a8fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.323392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb359ef3-0499-49e4-ae81-ea3de86a8fbb admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.324079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb359ef3-0499-49e4-ae81-ea3de86a8fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.326023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb359ef3-0499-49e4-ae81-ea3de86a8fbb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.327361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1a3df-6976-4478-9a90-afaed4e34ba9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.333742 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.334879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.335159 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.335864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] PATH_INFO: `/v3/projects/3f00924a742e4c61bcfcce3b5410bcbe` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.338099 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f00924a742e4c61bcfcce3b5410bcbe)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.340402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.356825 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 964/1938] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:39 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:39.356825 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb359ef3-0499-49e4-ae81-ea3de86a8fbb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.366541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb359ef3-0499-49e4-ae81-ea3de86a8fbb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.374108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22644860-b29f-4316-8bf4-b6ec74c08490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.375118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22644860-b29f-4316-8bf4-b6ec74c08490 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.375620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22644860-b29f-4316-8bf4-b6ec74c08490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.376215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22644860-b29f-4316-8bf4-b6ec74c08490 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.377531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22644860-b29f-4316-8bf4-b6ec74c08490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.382116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22644860-b29f-4316-8bf4-b6ec74c08490 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.407827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f00924a742e4c61bcfcce3b5410bcbe', 'request_id': 'req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:39.415709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 965/1939] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:39 2022] POST /identity/v3/users => generated 302 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:39.416541 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f00924a742e4c61bcfcce3b5410bcbe', 'request_id': 'req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:39.419968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22644860-b29f-4316-8bf4-b6ec74c08490 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.425787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f00924a742e4c61bcfcce3b5410bcbe', 'request_id': 'req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:39.431684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22644860-b29f-4316-8bf4-b6ec74c08490 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.439698 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] The token cache is being invalidate because project 3f00924a742e4c61bcfcce3b5410bcbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:39.440046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:39.440724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:39.442600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 975/1940] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:39 2022] DELETE /identity/v3/projects/3f00924a742e4c61bcfcce3b5410bcbe => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:39.446222 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.447648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:39.512071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6e2c5-d5a4-44b8-9294-4dfbd338e5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.512796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6e2c5-d5a4-44b8-9294-4dfbd338e5ed admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.513192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6e2c5-d5a4-44b8-9294-4dfbd338e5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.513472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6e2c5-d5a4-44b8-9294-4dfbd338e5ed admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.514221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6e2c5-d5a4-44b8-9294-4dfbd338e5ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.516167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6e2c5-d5a4-44b8-9294-4dfbd338e5ed admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.523705 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:39.541597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d81e13-6d7d-4a55-b3e9-41bad3a5cbd7 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.549853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 966/1941] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:39 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:39.549853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6e2c5-d5a4-44b8-9294-4dfbd338e5ed admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.549853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-95d6e2c5-d5a4-44b8-9294-4dfbd338e5ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:39.550583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270eabd-82ce-4cdb-afa4-555e6e472df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.551332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270eabd-82ce-4cdb-afa4-555e6e472df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.551623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270eabd-82ce-4cdb-afa4-555e6e472df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.551910 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270eabd-82ce-4cdb-afa4-555e6e472df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.552834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270eabd-82ce-4cdb-afa4-555e6e472df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.554900 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270eabd-82ce-4cdb-afa4-555e6e472df0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.562062 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6e2c5-d5a4-44b8-9294-4dfbd338e5ed admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.569001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165448a-e125-4e9e-ab5a-eb10c32111a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.569694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165448a-e125-4e9e-ab5a-eb10c32111a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.570049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165448a-e125-4e9e-ab5a-eb10c32111a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.570390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165448a-e125-4e9e-ab5a-eb10c32111a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.571476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 976/1942] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:39.571597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165448a-e125-4e9e-ab5a-eb10c32111a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.572223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270eabd-82ce-4cdb-afa4-555e6e472df0 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.573243 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165448a-e125-4e9e-ab5a-eb10c32111a0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.574292 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9270eabd-82ce-4cdb-afa4-555e6e472df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:39.584916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 967/1943] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:39.585936 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165448a-e125-4e9e-ab5a-eb10c32111a0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.589388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3165448a-e125-4e9e-ab5a-eb10c32111a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:39.594423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270eabd-82ce-4cdb-afa4-555e6e472df0 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.599930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.601145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.601509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.601909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] PATH_INFO: `/v3/users/3121c7e90d3643b7af969170ef21b73b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.603337 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3121c7e90d3643b7af969170ef21b73b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.605402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.606963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165448a-e125-4e9e-ab5a-eb10c32111a0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.611885 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef90845-1f47-41de-ba05-fb9ef0e4cc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.612564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef90845-1f47-41de-ba05-fb9ef0e4cc6c admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.612848 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef90845-1f47-41de-ba05-fb9ef0e4cc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.613126 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef90845-1f47-41de-ba05-fb9ef0e4cc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.613976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef90845-1f47-41de-ba05-fb9ef0e4cc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.615581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef90845-1f47-41de-ba05-fb9ef0e4cc6c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.631298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 968/1944] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:39.631298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef90845-1f47-41de-ba05-fb9ef0e4cc6c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.662621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef90845-1f47-41de-ba05-fb9ef0e4cc6c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.672793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e65cd9-46a4-47da-9cd0-a8fce2dbdc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.673748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e65cd9-46a4-47da-9cd0-a8fce2dbdc95 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.674159 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e65cd9-46a4-47da-9cd0-a8fce2dbdc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.674759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e65cd9-46a4-47da-9cd0-a8fce2dbdc95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.677971 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e65cd9-46a4-47da-9cd0-a8fce2dbdc95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.682423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e65cd9-46a4-47da-9cd0-a8fce2dbdc95 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.700429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:39.702999 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3121c7e90d3643b7af969170ef21b73b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:39.703167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 969/1945] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:39.703988 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e65cd9-46a4-47da-9cd0-a8fce2dbdc95 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.704378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3121c7e90d3643b7af969170ef21b73b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:39.706246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b8e65cd9-46a4-47da-9cd0-a8fce2dbdc95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:39.724546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3121c7e90d3643b7af969170ef21b73b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:39.726841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e65cd9-46a4-47da-9cd0-a8fce2dbdc95 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.732571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf137c-f98a-4860-8dd1-61596bab613d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.733701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf137c-f98a-4860-8dd1-61596bab613d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.734017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf137c-f98a-4860-8dd1-61596bab613d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.734299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf137c-f98a-4860-8dd1-61596bab613d admin admin] PATH_INFO: `/v3/projects/fed0a92759f9474c84c99f6ea30c15d5/users/f236e06841aa4be895f4d98c2ec3c786/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.742527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3121c7e90d3643b7af969170ef21b73b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:39.745805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3121c7e90d3643b7af969170ef21b73b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:39.762487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf137c-f98a-4860-8dd1-61596bab613d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fed0a92759f9474c84c99f6ea30c15d5, user_id=f236e06841aa4be895f4d98c2ec3c786, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.765183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf137c-f98a-4860-8dd1-61596bab613d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.784928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 977/1946] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:39 2022] DELETE /identity/v3/users/3121c7e90d3643b7af969170ef21b73b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:39.786712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.790598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 970/1947] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:39 2022] PUT /identity/v3/projects/fed0a92759f9474c84c99f6ea30c15d5/users/f236e06841aa4be895f4d98c2ec3c786/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:39.791300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:39.791876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf137c-f98a-4860-8dd1-61596bab613d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.796120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3ecf137c-f98a-4860-8dd1-61596bab613d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:39.905097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf137c-f98a-4860-8dd1-61596bab613d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.932654 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2100a7-d873-4ab9-8252-1513c1491f1c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:39.941338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d0b09-98d4-4902-9230-0ca57510db7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:39.942121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d0b09-98d4-4902-9230-0ca57510db7b admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:39.942413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d0b09-98d4-4902-9230-0ca57510db7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:39.943082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d0b09-98d4-4902-9230-0ca57510db7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:39.943833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d0b09-98d4-4902-9230-0ca57510db7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:39.945822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d0b09-98d4-4902-9230-0ca57510db7b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:39.977659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 978/1948] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:39.978363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d0b09-98d4-4902-9230-0ca57510db7b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:39.990736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d0b09-98d4-4902-9230-0ca57510db7b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:40.014257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec03153-5d33-4a5e-a8bb-d611d6af1e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:40.015032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec03153-5d33-4a5e-a8bb-d611d6af1e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.015489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec03153-5d33-4a5e-a8bb-d611d6af1e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.015956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec03153-5d33-4a5e-a8bb-d611d6af1e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.018643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec03153-5d33-4a5e-a8bb-d611d6af1e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:40.022189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec03153-5d33-4a5e-a8bb-d611d6af1e24 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:40.053651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 979/1949] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:40.053651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec03153-5d33-4a5e-a8bb-d611d6af1e24 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:40.070693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977a9e2-8608-47d7-b07b-a9b457dd4be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:40.071961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977a9e2-8608-47d7-b07b-a9b457dd4be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.072429 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977a9e2-8608-47d7-b07b-a9b457dd4be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.073188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977a9e2-8608-47d7-b07b-a9b457dd4be5 admin admin] PATH_INFO: `/v3/projects/d11acdd77f9840f984b16bd64b67eeea/users/22d4a830c43f477ebd93beb22711d267/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.077356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec03153-5d33-4a5e-a8bb-d611d6af1e24 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:40.083674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05df0e-7202-4a73-b9d7-de901a378992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:40.084715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05df0e-7202-4a73-b9d7-de901a378992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.085073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05df0e-7202-4a73-b9d7-de901a378992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.085361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05df0e-7202-4a73-b9d7-de901a378992 admin admin] PATH_INFO: `/v3/projects/d9ac72920cb24fed93fe20872091d35d/users/9fe2910ef51c4bd295547afc11c8d00a/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.097476 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977a9e2-8608-47d7-b07b-a9b457dd4be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11acdd77f9840f984b16bd64b67eeea, user_id=22d4a830c43f477ebd93beb22711d267, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:40.100682 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977a9e2-8608-47d7-b07b-a9b457dd4be5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:40.119147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 971/1950] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:39 2022] PUT /identity/v3/projects/d11acdd77f9840f984b16bd64b67eeea/users/22d4a830c43f477ebd93beb22711d267/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:40.120213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977a9e2-8608-47d7-b07b-a9b457dd4be5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:40.151400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05df0e-7202-4a73-b9d7-de901a378992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ac72920cb24fed93fe20872091d35d, user_id=9fe2910ef51c4bd295547afc11c8d00a, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:40.154532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05df0e-7202-4a73-b9d7-de901a378992 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:40.182430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 980/1951] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:40 2022] PUT /identity/v3/projects/d9ac72920cb24fed93fe20872091d35d/users/9fe2910ef51c4bd295547afc11c8d00a/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:40.184408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e258f-0f81-4d14-9a8f-f323410f21ba None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.184713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e258f-0f81-4d14-9a8f-f323410f21ba None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.184994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e258f-0f81-4d14-9a8f-f323410f21ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.194030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:40.238439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-892e258f-0f81-4d14-9a8f-f323410f21ba None None] MFA Rules not processed for user `22d4a830c43f477ebd93beb22711d267`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:40.239991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-892e258f-0f81-4d14-9a8f-f323410f21ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:40.258564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977a9e2-8608-47d7-b07b-a9b457dd4be5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:40.352883 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d178b2-6f1e-46b4-8f8d-c1ff45cc948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:40.353275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d178b2-6f1e-46b4-8f8d-c1ff45cc948b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.353499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d178b2-6f1e-46b4-8f8d-c1ff45cc948b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.354132 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d178b2-6f1e-46b4-8f8d-c1ff45cc948b admin admin] PATH_INFO: `/v3/projects/fed0a92759f9474c84c99f6ea30c15d5/users/f236e06841aa4be895f4d98c2ec3c786/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.355739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d178b2-6f1e-46b4-8f8d-c1ff45cc948b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fed0a92759f9474c84c99f6ea30c15d5, user_id=f236e06841aa4be895f4d98c2ec3c786, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:40.359135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d178b2-6f1e-46b4-8f8d-c1ff45cc948b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:40.365977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 981/1952] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:40.366851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6575da-7e88-41cd-900b-21c73c277cde None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.367198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6575da-7e88-41cd-900b-21c73c277cde None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.367480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6575da-7e88-41cd-900b-21c73c277cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.373802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 972/1953] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:40 2022] PUT /identity/v3/projects/fed0a92759f9474c84c99f6ea30c15d5/users/f236e06841aa4be895f4d98c2ec3c786/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:40.375176 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d178b2-6f1e-46b4-8f8d-c1ff45cc948b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:40.387140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a6d178b2-6f1e-46b4-8f8d-c1ff45cc948b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:40.401523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:40.405109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-fe6575da-7e88-41cd-900b-21c73c277cde None None] MFA Rules not processed for user `9fe2910ef51c4bd295547afc11c8d00a`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:40.407396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fe6575da-7e88-41cd-900b-21c73c277cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:40.458694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d178b2-6f1e-46b4-8f8d-c1ff45cc948b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:40.463863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:40.464911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.465201 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.465474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] PATH_INFO: `/v3/projects/bd8f65b210554291acf018cabc185fe6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.466888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd8f65b210554291acf018cabc185fe6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:40.469306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:40.476780 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 982/1954] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:40.477919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6575da-7e88-41cd-900b-21c73c277cde None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:40.489317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6575da-7e88-41cd-900b-21c73c277cde None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:40.493977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62aa8b9-d412-4abc-b57a-59957eb4822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:40.494647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62aa8b9-d412-4abc-b57a-59957eb4822c admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.494931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62aa8b9-d412-4abc-b57a-59957eb4822c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.495256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62aa8b9-d412-4abc-b57a-59957eb4822c admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.495993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62aa8b9-d412-4abc-b57a-59957eb4822c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:40.497938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62aa8b9-d412-4abc-b57a-59957eb4822c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:40.527484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd8f65b210554291acf018cabc185fe6', 'request_id': 'req-1af30167-8912-47fb-ae3e-b56d25ec4f8c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:40.529940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 983/1955] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:40 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:40.531406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd171597-1cfe-4cea-9c7d-0418e07df751 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.531686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd171597-1cfe-4cea-9c7d-0418e07df751 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.531963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd171597-1cfe-4cea-9c7d-0418e07df751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.539504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd8f65b210554291acf018cabc185fe6', 'request_id': 'req-1af30167-8912-47fb-ae3e-b56d25ec4f8c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:40.547006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd8f65b210554291acf018cabc185fe6', 'request_id': 'req-1af30167-8912-47fb-ae3e-b56d25ec4f8c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:40.552861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:40.562766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] The token cache is being invalidate because project bd8f65b210554291acf018cabc185fe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:40.563056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:40.564865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:40.566909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 973/1956] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:40 2022] DELETE /identity/v3/projects/bd8f65b210554291acf018cabc185fe6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:40.567280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-cd171597-1cfe-4cea-9c7d-0418e07df751 None None] MFA Rules not processed for user `f236e06841aa4be895f4d98c2ec3c786`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:40.567660 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:40.568753 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cd171597-1cfe-4cea-9c7d-0418e07df751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:40.569089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:40.652960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af30167-8912-47fb-ae3e-b56d25ec4f8c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:40.661294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 984/1957] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:40.662501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd171597-1cfe-4cea-9c7d-0418e07df751 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:40.663876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cd171597-1cfe-4cea-9c7d-0418e07df751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:40.730613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc11023-3a5a-4dce-8102-bd820280c7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:40.731646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc11023-3a5a-4dce-8102-bd820280c7ae admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.731951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc11023-3a5a-4dce-8102-bd820280c7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.732428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc11023-3a5a-4dce-8102-bd820280c7ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.733046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc11023-3a5a-4dce-8102-bd820280c7ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:40.736048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc11023-3a5a-4dce-8102-bd820280c7ae admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:40.755485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd171597-1cfe-4cea-9c7d-0418e07df751 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:40.761470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c11c1-019d-49a4-a1fd-1aa2a0204a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:40.762230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c11c1-019d-49a4-a1fd-1aa2a0204a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.762514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c11c1-019d-49a4-a1fd-1aa2a0204a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.762789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c11c1-019d-49a4-a1fd-1aa2a0204a84 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.763491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c11c1-019d-49a4-a1fd-1aa2a0204a84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:40.765436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c11c1-019d-49a4-a1fd-1aa2a0204a84 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:40.771095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 974/1958] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:40 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:40.771095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45e69f-8cc3-42b8-b401-dfd83e4758a2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.771095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45e69f-8cc3-42b8-b401-dfd83e4758a2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.771095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45e69f-8cc3-42b8-b401-dfd83e4758a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.778605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:40.791121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4c45e69f-8cc3-42b8-b401-dfd83e4758a2 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:40.791360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 985/1959] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:40 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:40.792319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4c45e69f-8cc3-42b8-b401-dfd83e4758a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:40.792476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c11c1-019d-49a4-a1fd-1aa2a0204a84 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:40.795336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-458c11c1-019d-49a4-a1fd-1aa2a0204a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:40.857430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c11c1-019d-49a4-a1fd-1aa2a0204a84 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:40.865846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 975/1960] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:40.865846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45e69f-8cc3-42b8-b401-dfd83e4758a2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:40.873278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45e69f-8cc3-42b8-b401-dfd83e4758a2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:40.879537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b570a09-7edb-4a41-b02a-0ea3537aa3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:40.880256 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b570a09-7edb-4a41-b02a-0ea3537aa3ab admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.880543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b570a09-7edb-4a41-b02a-0ea3537aa3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.880870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b570a09-7edb-4a41-b02a-0ea3537aa3ab admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.881608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b570a09-7edb-4a41-b02a-0ea3537aa3ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:40.883651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b570a09-7edb-4a41-b02a-0ea3537aa3ab admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:40.909151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 976/1961] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:40 2022] POST /identity/v3/users => generated 297 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:40.910042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b570a09-7edb-4a41-b02a-0ea3537aa3ab admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:40.922483 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b570a09-7edb-4a41-b02a-0ea3537aa3ab admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:40.927810 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4717c-07c4-4e77-9d31-a7032ac63c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:40.929190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4717c-07c4-4e77-9d31-a7032ac63c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.929190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4717c-07c4-4e77-9d31-a7032ac63c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.929547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4717c-07c4-4e77-9d31-a7032ac63c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.930759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4717c-07c4-4e77-9d31-a7032ac63c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:40.931281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba084d8f-0b8b-4866-b54c-cb53f2175d0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:40.934382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba084d8f-0b8b-4866-b54c-cb53f2175d0c service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.934382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba084d8f-0b8b-4866-b54c-cb53f2175d0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.934609 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4717c-07c4-4e77-9d31-a7032ac63c15 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:40.934698 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba084d8f-0b8b-4866-b54c-cb53f2175d0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.935733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:40.947862 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 977/1962] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:40.949570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff1224-a9db-4d1d-bcda-1add4ea0e4a6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:40.953514 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff1224-a9db-4d1d-bcda-1add4ea0e4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:40.953918 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff1224-a9db-4d1d-bcda-1add4ea0e4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:40.988271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3fff1224-a9db-4d1d-bcda-1add4ea0e4a6 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:40.989981 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3fff1224-a9db-4d1d-bcda-1add4ea0e4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:41.000280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 978/1963] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:41.000965 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff1224-a9db-4d1d-bcda-1add4ea0e4a6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.014874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff1224-a9db-4d1d-bcda-1add4ea0e4a6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.021730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe36d07-d3e9-448f-9203-8e0405ac60cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.022774 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe36d07-d3e9-448f-9203-8e0405ac60cd admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.023131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe36d07-d3e9-448f-9203-8e0405ac60cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.023510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe36d07-d3e9-448f-9203-8e0405ac60cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.024326 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe36d07-d3e9-448f-9203-8e0405ac60cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.026192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe36d07-d3e9-448f-9203-8e0405ac60cd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.027332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba084d8f-0b8b-4866-b54c-cb53f2175d0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.031585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba084d8f-0b8b-4866-b54c-cb53f2175d0c service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.043759 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 986/1964] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:41.044733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba084d8f-0b8b-4866-b54c-cb53f2175d0c service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.059706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba084d8f-0b8b-4866-b54c-cb53f2175d0c service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.072487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 979/1965] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:41.073562 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe36d07-d3e9-448f-9203-8e0405ac60cd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.076652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3152b-ab31-4f06-8ab1-4eeac3b6d7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.077945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3152b-ab31-4f06-8ab1-4eeac3b6d7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.078311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3152b-ab31-4f06-8ab1-4eeac3b6d7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.078657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3152b-ab31-4f06-8ab1-4eeac3b6d7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.079803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3152b-ab31-4f06-8ab1-4eeac3b6d7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.082545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3152b-ab31-4f06-8ab1-4eeac3b6d7e8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.101123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe36d07-d3e9-448f-9203-8e0405ac60cd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.104510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 987/1966] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:41.104510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3152b-ab31-4f06-8ab1-4eeac3b6d7e8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.109181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efa04b-d7e3-459f-a251-4460bda5dd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.110007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efa04b-d7e3-459f-a251-4460bda5dd22 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.110362 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efa04b-d7e3-459f-a251-4460bda5dd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.110706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efa04b-d7e3-459f-a251-4460bda5dd22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.111768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efa04b-d7e3-459f-a251-4460bda5dd22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:41.112310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efa04b-d7e3-459f-a251-4460bda5dd22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.113899 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efa04b-d7e3-459f-a251-4460bda5dd22 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.116130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3152b-ab31-4f06-8ab1-4eeac3b6d7e8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.121958 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b261b7-fa1d-4da2-8b3c-d20fd78078b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.124940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b261b7-fa1d-4da2-8b3c-d20fd78078b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.125237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b261b7-fa1d-4da2-8b3c-d20fd78078b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.125522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b261b7-fa1d-4da2-8b3c-d20fd78078b8 admin admin] PATH_INFO: `/v3/projects/5dc9dc7be2514acc8814bfff7b0eed4c/users/2cb0b379bf094a4dac3c8c2c4d2fe33f/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.130561 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 980/1967] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:41.131347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efa04b-d7e3-459f-a251-4460bda5dd22 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.146386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efa04b-d7e3-459f-a251-4460bda5dd22 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.153963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1ffe8-042c-4165-b11e-76ce03c2757d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.153963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1ffe8-042c-4165-b11e-76ce03c2757d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.153963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1ffe8-042c-4165-b11e-76ce03c2757d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.153963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1ffe8-042c-4165-b11e-76ce03c2757d admin admin] PATH_INFO: `/v3/projects/4ad76210190e4c8f91c17d2369b0b8f0/users/839e75237c9346f68385de18406a831e/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.156624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b261b7-fa1d-4da2-8b3c-d20fd78078b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc9dc7be2514acc8814bfff7b0eed4c, user_id=2cb0b379bf094a4dac3c8c2c4d2fe33f, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.162841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b261b7-fa1d-4da2-8b3c-d20fd78078b8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.186777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1ffe8-042c-4165-b11e-76ce03c2757d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad76210190e4c8f91c17d2369b0b8f0, user_id=839e75237c9346f68385de18406a831e, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.190139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1ffe8-042c-4165-b11e-76ce03c2757d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.196125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 988/1968] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:41 2022] PUT /identity/v3/projects/5dc9dc7be2514acc8814bfff7b0eed4c/users/2cb0b379bf094a4dac3c8c2c4d2fe33f/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:41.197368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b261b7-fa1d-4da2-8b3c-d20fd78078b8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.212228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 981/1969] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:41 2022] PUT /identity/v3/projects/4ad76210190e4c8f91c17d2369b0b8f0/users/839e75237c9346f68385de18406a831e/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:41.213278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1ffe8-042c-4165-b11e-76ce03c2757d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.257391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:41.307122 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1ffe8-042c-4165-b11e-76ce03c2757d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.313473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98669d8-3e39-4b71-835e-6ce0c7bba05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.314239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98669d8-3e39-4b71-835e-6ce0c7bba05e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.314530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98669d8-3e39-4b71-835e-6ce0c7bba05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.314941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98669d8-3e39-4b71-835e-6ce0c7bba05e admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.315508 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98669d8-3e39-4b71-835e-6ce0c7bba05e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.317635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98669d8-3e39-4b71-835e-6ce0c7bba05e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.320653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b261b7-fa1d-4da2-8b3c-d20fd78078b8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.356469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 982/1970] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:41 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:41.357415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98669d8-3e39-4b71-835e-6ce0c7bba05e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.391484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa483049-fff8-4bf9-9a66-3914e84470c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.392503 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa483049-fff8-4bf9-9a66-3914e84470c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.392889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa483049-fff8-4bf9-9a66-3914e84470c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.393447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa483049-fff8-4bf9-9a66-3914e84470c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.395792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-aa483049-fff8-4bf9-9a66-3914e84470c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:41.436937 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98669d8-3e39-4b71-835e-6ce0c7bba05e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.442988 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9762f-c177-4560-afe1-5401812dad5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.443757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9762f-c177-4560-afe1-5401812dad5e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.444094 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9762f-c177-4560-afe1-5401812dad5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.444310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9762f-c177-4560-afe1-5401812dad5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.445055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9762f-c177-4560-afe1-5401812dad5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.447456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9762f-c177-4560-afe1-5401812dad5e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.461968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 983/1971] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:41.463407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a0e37-e5c4-4864-bc5e-3feb63e664a9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.463698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a0e37-e5c4-4864-bc5e-3feb63e664a9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.464904 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a0e37-e5c4-4864-bc5e-3feb63e664a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.480307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa483049-fff8-4bf9-9a66-3914e84470c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.482198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa483049-fff8-4bf9-9a66-3914e84470c0 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.494736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 989/1972] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:41.495821 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa483049-fff8-4bf9-9a66-3914e84470c0 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.510115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-356a0e37-e5c4-4864-bc5e-3feb63e664a9 None None] MFA Rules not processed for user `839e75237c9346f68385de18406a831e`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:41.511480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-356a0e37-e5c4-4864-bc5e-3feb63e664a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:41.516205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa483049-fff8-4bf9-9a66-3914e84470c0 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.522575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd495f-7366-45f9-b73f-55c862f3aa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.523789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd495f-7366-45f9-b73f-55c862f3aa59 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.524200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd495f-7366-45f9-b73f-55c862f3aa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.524573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd495f-7366-45f9-b73f-55c862f3aa59 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.525391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd495f-7366-45f9-b73f-55c862f3aa59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.528091 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd495f-7366-45f9-b73f-55c862f3aa59 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.531851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:41.562173 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 990/1973] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:41 2022] POST /identity/v3/users => generated 293 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:41.563342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd495f-7366-45f9-b73f-55c862f3aa59 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.576545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd495f-7366-45f9-b73f-55c862f3aa59 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.582138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f602a9f-e303-4246-b357-c412638a9ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.583127 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602a9f-e303-4246-b357-c412638a9ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.583319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602a9f-e303-4246-b357-c412638a9ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.583606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602a9f-e303-4246-b357-c412638a9ebe admin admin] PATH_INFO: `/v3/projects/5dc9dc7be2514acc8814bfff7b0eed4c/users/2cb0b379bf094a4dac3c8c2c4d2fe33f/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.592248 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f602a9f-e303-4246-b357-c412638a9ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc9dc7be2514acc8814bfff7b0eed4c, user_id=2cb0b379bf094a4dac3c8c2c4d2fe33f, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.595071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f602a9f-e303-4246-b357-c412638a9ebe admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.605799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 984/1974] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:41.606542 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a0e37-e5c4-4864-bc5e-3feb63e664a9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.612383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 991/1975] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:41 2022] PUT /identity/v3/projects/5dc9dc7be2514acc8814bfff7b0eed4c/users/2cb0b379bf094a4dac3c8c2c4d2fe33f/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:41.613347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f602a9f-e303-4246-b357-c412638a9ebe admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.617916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a0e37-e5c4-4864-bc5e-3feb63e664a9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.685584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa97521-0537-4408-8790-c7158dc81fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.686309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa97521-0537-4408-8790-c7158dc81fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.686609 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa97521-0537-4408-8790-c7158dc81fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.686877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa97521-0537-4408-8790-c7158dc81fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.687679 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa97521-0537-4408-8790-c7158dc81fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.689250 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa97521-0537-4408-8790-c7158dc81fa6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.691495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f602a9f-e303-4246-b357-c412638a9ebe admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.697194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e20d5-43fd-426d-9904-3d781edab0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.698418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e20d5-43fd-426d-9904-3d781edab0ec admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.698841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e20d5-43fd-426d-9904-3d781edab0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.699299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e20d5-43fd-426d-9904-3d781edab0ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.700376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e20d5-43fd-426d-9904-3d781edab0ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.703909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 985/1976] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:41.704060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e20d5-43fd-426d-9904-3d781edab0ec admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.705458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9533ee-03b1-4ed1-9563-ae47b5efaad7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.705851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9533ee-03b1-4ed1-9563-ae47b5efaad7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.706258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9533ee-03b1-4ed1-9563-ae47b5efaad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.715234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:41.739468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ad9533ee-03b1-4ed1-9563-ae47b5efaad7 None None] MFA Rules not processed for user `2cb0b379bf094a4dac3c8c2c4d2fe33f`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:41.739712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 992/1977] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:41 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:41.739798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e20d5-43fd-426d-9904-3d781edab0ec admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.742357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ad9533ee-03b1-4ed1-9563-ae47b5efaad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:41.820757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e20d5-43fd-426d-9904-3d781edab0ec admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.827927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342adc6c-373e-4f8d-9eb0-900d5e631c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.828623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342adc6c-373e-4f8d-9eb0-900d5e631c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.828952 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342adc6c-373e-4f8d-9eb0-900d5e631c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.829238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342adc6c-373e-4f8d-9eb0-900d5e631c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.831533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342adc6c-373e-4f8d-9eb0-900d5e631c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.833579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342adc6c-373e-4f8d-9eb0-900d5e631c58 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.837758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 986/1978] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:41.838711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9533ee-03b1-4ed1-9563-ae47b5efaad7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.853262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9533ee-03b1-4ed1-9563-ae47b5efaad7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.859788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cf741-3690-4fa9-aabf-b5dd08bc4a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.860558 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cf741-3690-4fa9-aabf-b5dd08bc4a18 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.860944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cf741-3690-4fa9-aabf-b5dd08bc4a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.861428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cf741-3690-4fa9-aabf-b5dd08bc4a18 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.862763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cf741-3690-4fa9-aabf-b5dd08bc4a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.863381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 993/1979] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:41.864428 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342adc6c-373e-4f8d-9eb0-900d5e631c58 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.864891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cf741-3690-4fa9-aabf-b5dd08bc4a18 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.894014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 987/1980] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:41 2022] POST /identity/v3/users => generated 296 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:41.895107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cf741-3690-4fa9-aabf-b5dd08bc4a18 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:41.907975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cf741-3690-4fa9-aabf-b5dd08bc4a18 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.915280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c7010-c4b9-471c-889a-8e8327151569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:41.916245 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c7010-c4b9-471c-889a-8e8327151569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:41.916529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c7010-c4b9-471c-889a-8e8327151569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:41.916806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c7010-c4b9-471c-889a-8e8327151569 admin admin] PATH_INFO: `/v3/projects/49293146ea2f4fbd9c457a821f567991/users/185b640f6d034bdeaee4d053b145eecf/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:41.942560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c7010-c4b9-471c-889a-8e8327151569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49293146ea2f4fbd9c457a821f567991, user_id=185b640f6d034bdeaee4d053b145eecf, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:41.946143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c7010-c4b9-471c-889a-8e8327151569 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:41.951515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:41.967066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342adc6c-373e-4f8d-9eb0-900d5e631c58 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:41.972485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 988/1981] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:41 2022] PUT /identity/v3/projects/49293146ea2f4fbd9c457a821f567991/users/185b640f6d034bdeaee4d053b145eecf/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:41.974030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c7010-c4b9-471c-889a-8e8327151569 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:42.029993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6829650-fca3-4ebf-b0aa-f2e2cc4850a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:42.030814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6829650-fca3-4ebf-b0aa-f2e2cc4850a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:42.031101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6829650-fca3-4ebf-b0aa-f2e2cc4850a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:42.031398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6829650-fca3-4ebf-b0aa-f2e2cc4850a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:42.048818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c7010-c4b9-471c-889a-8e8327151569 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:42.053727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28652411-f3a2-40ba-acec-142533595022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:42.054437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28652411-f3a2-40ba-acec-142533595022 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:42.054637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28652411-f3a2-40ba-acec-142533595022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:42.054914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28652411-f3a2-40ba-acec-142533595022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:42.055678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28652411-f3a2-40ba-acec-142533595022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:42.057356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28652411-f3a2-40ba-acec-142533595022 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:42.068446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 989/1982] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:42.070996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab3732-a012-4050-a1b6-66728c5db6c6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:42.070996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab3732-a012-4050-a1b6-66728c5db6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:42.070996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab3732-a012-4050-a1b6-66728c5db6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:42.100118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:42.102003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-97ab3732-a012-4050-a1b6-66728c5db6c6 None None] MFA Rules not processed for user `185b640f6d034bdeaee4d053b145eecf`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:42.103662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-97ab3732-a012-4050-a1b6-66728c5db6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:42.104228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6829650-fca3-4ebf-b0aa-f2e2cc4850a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:42.106012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6829650-fca3-4ebf-b0aa-f2e2cc4850a6 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:42.118425 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 994/1983] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:42.119772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6829650-fca3-4ebf-b0aa-f2e2cc4850a6 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:42.131073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6829650-fca3-4ebf-b0aa-f2e2cc4850a6 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:42.137002 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab9da6-11fc-4e17-8040-002ac64cad73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:42.138100 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9da6-11fc-4e17-8040-002ac64cad73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:42.138357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9da6-11fc-4e17-8040-002ac64cad73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:42.138640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9da6-11fc-4e17-8040-002ac64cad73 admin admin] PATH_INFO: `/v3/projects/1c4c0e33dccf4b20b5de9807afdc6535/users/a06c04db53d0464ba79fba5cb56c0496/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:42.155404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab9da6-11fc-4e17-8040-002ac64cad73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4c0e33dccf4b20b5de9807afdc6535, user_id=a06c04db53d0464ba79fba5cb56c0496, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:42.158213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab9da6-11fc-4e17-8040-002ac64cad73 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:42.176604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 995/1984] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:42 2022] PUT /identity/v3/projects/1c4c0e33dccf4b20b5de9807afdc6535/users/a06c04db53d0464ba79fba5cb56c0496/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:42.178776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab9da6-11fc-4e17-8040-002ac64cad73 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:42.185506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 990/1985] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:42.197645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab3732-a012-4050-a1b6-66728c5db6c6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:42.210855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab3732-a012-4050-a1b6-66728c5db6c6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:42.263558 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab9da6-11fc-4e17-8040-002ac64cad73 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:42.269519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c398bb-e3eb-49c3-96e8-0ee4c390bad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:42.270269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c398bb-e3eb-49c3-96e8-0ee4c390bad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:42.270552 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c398bb-e3eb-49c3-96e8-0ee4c390bad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:42.270824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c398bb-e3eb-49c3-96e8-0ee4c390bad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:42.271580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c398bb-e3eb-49c3-96e8-0ee4c390bad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:42.273319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c398bb-e3eb-49c3-96e8-0ee4c390bad0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:42.294950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0f35d-ffc0-4fe9-82c6-6d485419e852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:42.296224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0f35d-ffc0-4fe9-82c6-6d485419e852 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:42.296599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0f35d-ffc0-4fe9-82c6-6d485419e852 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:42.297083 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0f35d-ffc0-4fe9-82c6-6d485419e852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:42.303306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 996/1986] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:15:42.307503 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c398bb-e3eb-49c3-96e8-0ee4c390bad0 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:42.320741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c398bb-e3eb-49c3-96e8-0ee4c390bad0 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:42.347008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c771172-6965-4aa9-9d29-7e0b556b6e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:42.347955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c771172-6965-4aa9-9d29-7e0b556b6e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:42.348243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c771172-6965-4aa9-9d29-7e0b556b6e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:42.348524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c771172-6965-4aa9-9d29-7e0b556b6e8b admin admin] PATH_INFO: `/v3/projects/1c4c0e33dccf4b20b5de9807afdc6535/users/a06c04db53d0464ba79fba5cb56c0496/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:42.350721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c771172-6965-4aa9-9d29-7e0b556b6e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4c0e33dccf4b20b5de9807afdc6535, user_id=a06c04db53d0464ba79fba5cb56c0496, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:42.353602 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c771172-6965-4aa9-9d29-7e0b556b6e8b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:42.385442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 997/1987] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:42 2022] PUT /identity/v3/projects/1c4c0e33dccf4b20b5de9807afdc6535/users/a06c04db53d0464ba79fba5cb56c0496/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:42.413265 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d00d4e-3225-4729-86b1-3e4d009b4a97 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:42.416743 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d00d4e-3225-4729-86b1-3e4d009b4a97 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:42.417443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d00d4e-3225-4729-86b1-3e4d009b4a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:42.417641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:42.473169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0f35d-ffc0-4fe9-82c6-6d485419e852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:42.474611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0f35d-ffc0-4fe9-82c6-6d485419e852 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:42.489625 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 991/1988] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:15:42.505004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e0d00d4e-3225-4729-86b1-3e4d009b4a97 None None] MFA Rules not processed for user `a06c04db53d0464ba79fba5cb56c0496`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:42.506565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e0d00d4e-3225-4729-86b1-3e4d009b4a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:42.671689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 998/1989] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:42.711403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0f35d-ffc0-4fe9-82c6-6d485419e852 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:42.744454 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d00d4e-3225-4729-86b1-3e4d009b4a97 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:42.747269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e0d00d4e-3225-4729-86b1-3e4d009b4a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:42.831095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0f35d-ffc0-4fe9-82c6-6d485419e852 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:42.868076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:42.906777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d00d4e-3225-4729-86b1-3e4d009b4a97 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:42.912382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab455cba-ed06-40f1-828b-afe0b820b637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:42.913225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab455cba-ed06-40f1-828b-afe0b820b637 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:42.913490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab455cba-ed06-40f1-828b-afe0b820b637 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:42.913953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab455cba-ed06-40f1-828b-afe0b820b637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:42.930582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab455cba-ed06-40f1-828b-afe0b820b637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:42.932286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab455cba-ed06-40f1-828b-afe0b820b637 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:42.968543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 999/1990] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:42.971622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aa9b1-8348-4f5b-af9c-901bb5886b88 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb0b379bf094a4dac3c8c2c4d2fe33f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dc9dc7be2514acc8814bfff7b0eed4c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:42.972545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aa9b1-8348-4f5b-af9c-901bb5886b88 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:42.972836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aa9b1-8348-4f5b-af9c-901bb5886b88 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:42.973190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aa9b1-8348-4f5b-af9c-901bb5886b88 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:42.973943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5aa9b1-8348-4f5b-af9c-901bb5886b88 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:42.974259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5aa9b1-8348-4f5b-af9c-901bb5886b88 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:42.975942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5aa9b1-8348-4f5b-af9c-901bb5886b88 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:42.996738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab455cba-ed06-40f1-828b-afe0b820b637 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:42.997256 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 992/1991] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:43.018013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab455cba-ed06-40f1-828b-afe0b820b637 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:43.024732 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204c7c2-c6c7-48d0-a77c-d5258b12e04f tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb0b379bf094a4dac3c8c2c4d2fe33f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dc9dc7be2514acc8814bfff7b0eed4c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:43.025713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204c7c2-c6c7-48d0-a77c-d5258b12e04f tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:43.026039 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204c7c2-c6c7-48d0-a77c-d5258b12e04f tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:43.026333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204c7c2-c6c7-48d0-a77c-d5258b12e04f tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:43.027071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204c7c2-c6c7-48d0-a77c-d5258b12e04f tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:43.029251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204c7c2-c6c7-48d0-a77c-d5258b12e04f tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:43.076232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aa9b1-8348-4f5b-af9c-901bb5886b88 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:43.078009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1000/1992] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:42 2022] POST /identity/v3/projects => generated 354 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:43.100992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204c7c2-c6c7-48d0-a77c-d5258b12e04f tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:43.104429 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aa9b1-8348-4f5b-af9c-901bb5886b88 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:43.213480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204c7c2-c6c7-48d0-a77c-d5258b12e04f tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:43.218701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4548d9-3264-4e54-b97a-63e91003c996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:43.219329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73642bd0-5dfd-4f37-8f4f-d07a4ed3c238 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb0b379bf094a4dac3c8c2c4d2fe33f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dc9dc7be2514acc8814bfff7b0eed4c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:43.219454 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4548d9-3264-4e54-b97a-63e91003c996 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:43.220341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73642bd0-5dfd-4f37-8f4f-d07a4ed3c238 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:43.220585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4548d9-3264-4e54-b97a-63e91003c996 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:43.220974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73642bd0-5dfd-4f37-8f4f-d07a4ed3c238 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:43.221100 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4548d9-3264-4e54-b97a-63e91003c996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:43.221509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73642bd0-5dfd-4f37-8f4f-d07a4ed3c238 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:43.222602 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73642bd0-5dfd-4f37-8f4f-d07a4ed3c238 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:43.223067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73642bd0-5dfd-4f37-8f4f-d07a4ed3c238 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:43.223936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ea4548d9-3264-4e54-b97a-63e91003c996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:43.225569 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73642bd0-5dfd-4f37-8f4f-d07a4ed3c238 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:43.242056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 993/1993] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:15:43.304945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4548d9-3264-4e54-b97a-63e91003c996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:43.306461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4548d9-3264-4e54-b97a-63e91003c996 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:43.316259 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1001/1994] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:43.379190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73642bd0-5dfd-4f37-8f4f-d07a4ed3c238 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:43.391425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73642bd0-5dfd-4f37-8f4f-d07a4ed3c238 tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:43.397452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb0b379bf094a4dac3c8c2c4d2fe33f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dc9dc7be2514acc8814bfff7b0eed4c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:43.398454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:43.398817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:43.399180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] PATH_INFO: `/v3/projects/149c6401c65f4b31b2c792e2f3055680` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:43.402137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: Authorizing `identity:delete_project(project_id=149c6401c65f4b31b2c792e2f3055680)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:43.404449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:43.407246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:43.508034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149c6401c65f4b31b2c792e2f3055680', 'request_id': 'req-4647f47d-6b54-4ca1-88fd-7131f11de76e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:43.521420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149c6401c65f4b31b2c792e2f3055680', 'request_id': 'req-4647f47d-6b54-4ca1-88fd-7131f11de76e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:43.532433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149c6401c65f4b31b2c792e2f3055680', 'request_id': 'req-4647f47d-6b54-4ca1-88fd-7131f11de76e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:43.561280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] The token cache is being invalidate because project 149c6401c65f4b31b2c792e2f3055680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:43.562590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:43.565006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4647f47d-6b54-4ca1-88fd-7131f11de76e tempest-VolumeQuotaClassesTest-2112953903 tempest-VolumeQuotaClassesTest-2112953903-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:43.575241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 994/1995] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:43 2022] DELETE /identity/v3/projects/149c6401c65f4b31b2c792e2f3055680 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:44.007082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4548d9-3264-4e54-b97a-63e91003c996 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:44.089668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4548d9-3264-4e54-b97a-63e91003c996 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:44.117396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:44.257331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:44.259631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:44.259952 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:44.260234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] PATH_INFO: `/v3/users/9fe2910ef51c4bd295547afc11c8d00a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:44.263572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe2910ef51c4bd295547afc11c8d00a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:44.266515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:44.355396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe2910ef51c4bd295547afc11c8d00a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:44.356893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe2910ef51c4bd295547afc11c8d00a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:44.373698 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe2910ef51c4bd295547afc11c8d00a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:44.391100 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe2910ef51c4bd295547afc11c8d00a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:44.392085 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe2910ef51c4bd295547afc11c8d00a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:44.413022 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1002/1996] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:44 2022] DELETE /identity/v3/users/9fe2910ef51c4bd295547afc11c8d00a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:44.417679 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d642196-3765-444b-a17b-1d2ff2b90347 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:44.418008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d642196-3765-444b-a17b-1d2ff2b90347 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:44.418341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d642196-3765-444b-a17b-1d2ff2b90347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:44.438054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0d642196-3765-444b-a17b-1d2ff2b90347 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:44.440790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0d642196-3765-444b-a17b-1d2ff2b90347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:44.561449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 995/1997] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:15:44.574811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:44.576404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:44.639280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a04c0a-c63f-4ea8-ac58-2cc17b522192 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:44.643718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d16d9-21da-467b-b96e-b262a3de26fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:44.644480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d16d9-21da-467b-b96e-b262a3de26fd service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:44.644825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d16d9-21da-467b-b96e-b262a3de26fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:44.645090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d16d9-21da-467b-b96e-b262a3de26fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:44.656401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d16d9-21da-467b-b96e-b262a3de26fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:44.658190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d16d9-21da-467b-b96e-b262a3de26fd service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:44.669757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1003/1998] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:45.056607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d642196-3765-444b-a17b-1d2ff2b90347 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:45.058161 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0d642196-3765-444b-a17b-1d2ff2b90347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:45.079724 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d642196-3765-444b-a17b-1d2ff2b90347 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:45.091237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:45.093340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:45.093632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:45.093927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] PATH_INFO: `/v3/projects/d9ac72920cb24fed93fe20872091d35d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:45.097673 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ac72920cb24fed93fe20872091d35d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:45.101285 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:45.203025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ac72920cb24fed93fe20872091d35d', 'request_id': 'req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:45.220211 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ac72920cb24fed93fe20872091d35d', 'request_id': 'req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:45.245746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ac72920cb24fed93fe20872091d35d', 'request_id': 'req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:45.262708 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] The token cache is being invalidate because project d9ac72920cb24fed93fe20872091d35d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:45.263004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:45.263674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:45.265460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 996/1999] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:45 2022] DELETE /identity/v3/projects/d9ac72920cb24fed93fe20872091d35d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:45.269915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d16d9-21da-467b-b96e-b262a3de26fd service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:45.280758 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d16d9-21da-467b-b96e-b262a3de26fd service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:45.338643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:45.368668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:45.370156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:45.370479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:45.370753 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] PATH_INFO: `/v3/users/2cb0b379bf094a4dac3c8c2c4d2fe33f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:45.372714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb0b379bf094a4dac3c8c2c4d2fe33f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:45.382345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:45.484011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb0b379bf094a4dac3c8c2c4d2fe33f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:45.484868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb0b379bf094a4dac3c8c2c4d2fe33f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:45.505534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb0b379bf094a4dac3c8c2c4d2fe33f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:45.527125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb0b379bf094a4dac3c8c2c4d2fe33f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:45.530460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb0b379bf094a4dac3c8c2c4d2fe33f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:45.575329 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1004/2000] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:45 2022] DELETE /identity/v3/users/2cb0b379bf094a4dac3c8c2c4d2fe33f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:45.920117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:45.922578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:45.939338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2141a2-e21f-43ac-b8a5-498f4f32b1dd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:45.942305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:45.944286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:45.971802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:46.044212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2c455-531c-4062-818e-1317e7413a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.045363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2c455-531c-4062-818e-1317e7413a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.045617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2c455-531c-4062-818e-1317e7413a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.045922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2c455-531c-4062-818e-1317e7413a53 admin admin] PATH_INFO: `/v3/users/7c7736d1595e41aa9ef207b777034096` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.047186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2c455-531c-4062-818e-1317e7413a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c7736d1595e41aa9ef207b777034096)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.049431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373cc29-5720-42ca-993f-1bcedc322cb0 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.050556 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2c455-531c-4062-818e-1317e7413a53 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.055351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.056561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.057061 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.057343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] PATH_INFO: `/v3/projects/5dc9dc7be2514acc8814bfff7b0eed4c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.058629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc9dc7be2514acc8814bfff7b0eed4c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.060708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.134884 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d3b2c455-531c-4062-818e-1317e7413a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c7736d1595e41aa9ef207b777034096'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.135893 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d3b2c455-531c-4062-818e-1317e7413a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c7736d1595e41aa9ef207b777034096'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.137566 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d3b2c455-531c-4062-818e-1317e7413a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c7736d1595e41aa9ef207b777034096'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.143353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc9dc7be2514acc8814bfff7b0eed4c', 'request_id': 'req-17c83068-cb61-4e9b-a498-009ae44a31b4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.157783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc9dc7be2514acc8814bfff7b0eed4c', 'request_id': 'req-17c83068-cb61-4e9b-a498-009ae44a31b4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.161294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d3b2c455-531c-4062-818e-1317e7413a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c7736d1595e41aa9ef207b777034096'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.166692 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc9dc7be2514acc8814bfff7b0eed4c', 'request_id': 'req-17c83068-cb61-4e9b-a498-009ae44a31b4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.169203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:46.173261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d3b2c455-531c-4062-818e-1317e7413a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c7736d1595e41aa9ef207b777034096'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.184317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] The token cache is being invalidate because project 5dc9dc7be2514acc8814bfff7b0eed4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:46.184809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.186388 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-17c83068-cb61-4e9b-a498-009ae44a31b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.188615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1005/2001] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:45 2022] DELETE /identity/v3/projects/5dc9dc7be2514acc8814bfff7b0eed4c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:46.194859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8a2d8-0bff-42c5-8a52-498eb3e4bd1a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.195196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8a2d8-0bff-42c5-8a52-498eb3e4bd1a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.195572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8a2d8-0bff-42c5-8a52-498eb3e4bd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.212422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 997/2002] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:45 2022] DELETE /identity/v3/users/7c7736d1595e41aa9ef207b777034096 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:46.216560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4583e3d-124f-4afb-8ab3-f52051929ec0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.217297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4583e3d-124f-4afb-8ab3-f52051929ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.217578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4583e3d-124f-4afb-8ab3-f52051929ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.222181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-57f8a2d8-0bff-42c5-8a52-498eb3e4bd1a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:46.223810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-57f8a2d8-0bff-42c5-8a52-498eb3e4bd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:46.235014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e4583e3d-124f-4afb-8ab3-f52051929ec0 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:46.236220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e4583e3d-124f-4afb-8ab3-f52051929ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:46.295263 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1006/2003] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:46.302802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd741c-5431-4182-805f-ddd967206216 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.303091 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd741c-5431-4182-805f-ddd967206216 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.303371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd741c-5431-4182-805f-ddd967206216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.309447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 998/2004] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:46.319359 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4583e3d-124f-4afb-8ab3-f52051929ec0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.320771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e4583e3d-124f-4afb-8ab3-f52051929ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:46.322309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-dfdd741c-5431-4182-805f-ddd967206216 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:46.323479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-dfdd741c-5431-4182-805f-ddd967206216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:46.339960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1007/2005] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:15:46.340942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd741c-5431-4182-805f-ddd967206216 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.352123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd741c-5431-4182-805f-ddd967206216 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.357276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee9988-153c-4a45-81af-f067f66612b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.357984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee9988-153c-4a45-81af-f067f66612b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.358326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee9988-153c-4a45-81af-f067f66612b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.358558 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee9988-153c-4a45-81af-f067f66612b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.359278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee9988-153c-4a45-81af-f067f66612b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:46.359585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee9988-153c-4a45-81af-f067f66612b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.361195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee9988-153c-4a45-81af-f067f66612b8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.367529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:46.375520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1008/2006] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:46.380107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee9988-153c-4a45-81af-f067f66612b8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.387351 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4583e3d-124f-4afb-8ab3-f52051929ec0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.390296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee9988-153c-4a45-81af-f067f66612b8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.391048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f7041-e75c-4716-88e8-1990267a908f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.391714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f7041-e75c-4716-88e8-1990267a908f service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.391997 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f7041-e75c-4716-88e8-1990267a908f service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.392273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f7041-e75c-4716-88e8-1990267a908f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.395272 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c97e5f-0ce5-4f6c-bfbc-0547b1b40679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.395951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97e5f-0ce5-4f6c-bfbc-0547b1b40679 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.396236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97e5f-0ce5-4f6c-bfbc-0547b1b40679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.396514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97e5f-0ce5-4f6c-bfbc-0547b1b40679 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.397208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c97e5f-0ce5-4f6c-bfbc-0547b1b40679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.399154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c97e5f-0ce5-4f6c-bfbc-0547b1b40679 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.401732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f7041-e75c-4716-88e8-1990267a908f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.403195 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f7041-e75c-4716-88e8-1990267a908f service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.413037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 999/2007] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:15:46.426601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1009/2008] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:46 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:15:46.430931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f7041-e75c-4716-88e8-1990267a908f service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.441464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f7041-e75c-4716-88e8-1990267a908f service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.502058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ce42a-6de2-4465-ac57-c2d847d05d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.502752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ce42a-6de2-4465-ac57-c2d847d05d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.503043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ce42a-6de2-4465-ac57-c2d847d05d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.503326 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ce42a-6de2-4465-ac57-c2d847d05d59 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.504030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ce42a-6de2-4465-ac57-c2d847d05d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.506008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ce42a-6de2-4465-ac57-c2d847d05d59 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.530374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1000/2009] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:46 2022] POST /identity/v3/users => generated 301 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:46.531658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c97e5f-0ce5-4f6c-bfbc-0547b1b40679 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.542396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c97e5f-0ce5-4f6c-bfbc-0547b1b40679 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.547277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44b14b-cd37-4b20-b18e-65bc5e1f15de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.547942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44b14b-cd37-4b20-b18e-65bc5e1f15de admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.548229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44b14b-cd37-4b20-b18e-65bc5e1f15de admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.548513 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44b14b-cd37-4b20-b18e-65bc5e1f15de admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.549321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44b14b-cd37-4b20-b18e-65bc5e1f15de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.551029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44b14b-cd37-4b20-b18e-65bc5e1f15de admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.563334 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1010/2010] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:15:46.566567 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ce42a-6de2-4465-ac57-c2d847d05d59 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.577783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ce42a-6de2-4465-ac57-c2d847d05d59 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.583610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6a649-dad0-4f1f-9094-d69de8053c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.584294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6a649-dad0-4f1f-9094-d69de8053c19 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.584576 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6a649-dad0-4f1f-9094-d69de8053c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.584859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6a649-dad0-4f1f-9094-d69de8053c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.585603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6a649-dad0-4f1f-9094-d69de8053c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.587757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6a649-dad0-4f1f-9094-d69de8053c19 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.602012 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1001/2011] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:46.604164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44b14b-cd37-4b20-b18e-65bc5e1f15de admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.626427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44b14b-cd37-4b20-b18e-65bc5e1f15de admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.633290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838037f-a90c-4b64-a812-0f3a58206cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.634595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838037f-a90c-4b64-a812-0f3a58206cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.634881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838037f-a90c-4b64-a812-0f3a58206cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.635163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838037f-a90c-4b64-a812-0f3a58206cbc admin admin] PATH_INFO: `/v3/projects/1079706b2e4848b192e1b674433d36cc/users/23943474dccc41b6bdd23d164d093278/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.656692 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838037f-a90c-4b64-a812-0f3a58206cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1079706b2e4848b192e1b674433d36cc, user_id=23943474dccc41b6bdd23d164d093278, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.659540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838037f-a90c-4b64-a812-0f3a58206cbc admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.677727 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1011/2012] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:46 2022] PUT /identity/v3/projects/1079706b2e4848b192e1b674433d36cc/users/23943474dccc41b6bdd23d164d093278/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:46.689431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6646fb-e949-4003-9cdd-6306cefdbc37 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.689431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6646fb-e949-4003-9cdd-6306cefdbc37 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.689637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6646fb-e949-4003-9cdd-6306cefdbc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.715294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838037f-a90c-4b64-a812-0f3a58206cbc admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.715294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7838037f-a90c-4b64-a812-0f3a58206cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:46.726867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2e6646fb-e949-4003-9cdd-6306cefdbc37 None None] MFA Rules not processed for user `23943474dccc41b6bdd23d164d093278`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:46.728095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2e6646fb-e949-4003-9cdd-6306cefdbc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:46.789313 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838037f-a90c-4b64-a812-0f3a58206cbc admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.794669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.794961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1002/2013] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:46.795487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.795774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.796051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] PATH_INFO: `/v3/projects/a1b0065ad771431784ce393d978942aa` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.797259 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b0065ad771431784ce393d978942aa)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.799368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.799496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6646fb-e949-4003-9cdd-6306cefdbc37 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.810664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6646fb-e949-4003-9cdd-6306cefdbc37 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.815608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dbe65-f924-42e4-9bc5-05cb9de2eba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.816275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dbe65-f924-42e4-9bc5-05cb9de2eba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.816564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dbe65-f924-42e4-9bc5-05cb9de2eba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.816844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dbe65-f924-42e4-9bc5-05cb9de2eba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.817223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:46.817543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dbe65-f924-42e4-9bc5-05cb9de2eba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.819716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dbe65-f924-42e4-9bc5-05cb9de2eba2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.847003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1003/2014] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:46 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:15:46.851147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dbe65-f924-42e4-9bc5-05cb9de2eba2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.853373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b0065ad771431784ce393d978942aa', 'request_id': 'req-9a430bf3-b35c-471d-afb4-00be2c071c11'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.866398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b0065ad771431784ce393d978942aa', 'request_id': 'req-9a430bf3-b35c-471d-afb4-00be2c071c11'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.877123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b0065ad771431784ce393d978942aa', 'request_id': 'req-9a430bf3-b35c-471d-afb4-00be2c071c11'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.893394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] The token cache is being invalidate because project a1b0065ad771431784ce393d978942aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:46.893693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.894586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:46.894699 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:46.896555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1012/2015] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:46 2022] DELETE /identity/v3/projects/a1b0065ad771431784ce393d978942aa => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:46.901142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.905293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:46.939101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dbe65-f924-42e4-9bc5-05cb9de2eba2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.944710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2811694-9f72-423f-b591-3036f69c1189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.946048 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2811694-9f72-423f-b591-3036f69c1189 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.946391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2811694-9f72-423f-b591-3036f69c1189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.946934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2811694-9f72-423f-b591-3036f69c1189 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.947524 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2811694-9f72-423f-b591-3036f69c1189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.949999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2811694-9f72-423f-b591-3036f69c1189 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.973173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1004/2016] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:46 2022] POST /identity/v3/users => generated 306 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:15:46.977057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2811694-9f72-423f-b591-3036f69c1189 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:46.978731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c2811694-9f72-423f-b591-3036f69c1189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:46.987668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a430bf3-b35c-471d-afb4-00be2c071c11 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.992628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2811694-9f72-423f-b591-3036f69c1189 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:46.992887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.993940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.994238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.994519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] PATH_INFO: `/v3/users/f236e06841aa4be895f4d98c2ec3c786` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.995740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f236e06841aa4be895f4d98c2ec3c786)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:46.997669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:46.997995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7297931-512b-4ba4-874f-a308b6908d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:46.998683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7297931-512b-4ba4-874f-a308b6908d22 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:46.998974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7297931-512b-4ba4-874f-a308b6908d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:46.999255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7297931-512b-4ba4-874f-a308b6908d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:46.999994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7297931-512b-4ba4-874f-a308b6908d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.001633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7297931-512b-4ba4-874f-a308b6908d22 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.013916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1005/2017] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:15:47.018315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7297931-512b-4ba4-874f-a308b6908d22 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.029105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7297931-512b-4ba4-874f-a308b6908d22 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.034580 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87d26f-be6b-4974-b6e1-50a0335a232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.035633 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87d26f-be6b-4974-b6e1-50a0335a232f admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.036033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87d26f-be6b-4974-b6e1-50a0335a232f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.036277 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87d26f-be6b-4974-b6e1-50a0335a232f admin admin] PATH_INFO: `/v3/projects/9431c7f47d38408a9431a57532e7468f/users/8a52f860d0484e2689cbfa1cb00b0236/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.053834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87d26f-be6b-4974-b6e1-50a0335a232f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9431c7f47d38408a9431a57532e7468f, user_id=8a52f860d0484e2689cbfa1cb00b0236, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.054950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f236e06841aa4be895f4d98c2ec3c786'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:47.055741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f236e06841aa4be895f4d98c2ec3c786'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:47.056612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87d26f-be6b-4974-b6e1-50a0335a232f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.065365 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f236e06841aa4be895f4d98c2ec3c786'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:47.072745 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1006/2018] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:47 2022] PUT /identity/v3/projects/9431c7f47d38408a9431a57532e7468f/users/8a52f860d0484e2689cbfa1cb00b0236/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:47.076475 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87d26f-be6b-4974-b6e1-50a0335a232f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.078356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bf87d26f-be6b-4974-b6e1-50a0335a232f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:47.081423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f236e06841aa4be895f4d98c2ec3c786'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:47.082202 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f236e06841aa4be895f4d98c2ec3c786'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:47.101376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1013/2019] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:46 2022] DELETE /identity/v3/users/f236e06841aa4be895f4d98c2ec3c786 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:47.146557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87d26f-be6b-4974-b6e1-50a0335a232f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.222259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307093dc-5df6-4172-a785-3a326bef95e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.223031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307093dc-5df6-4172-a785-3a326bef95e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.223403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307093dc-5df6-4172-a785-3a326bef95e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.223743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307093dc-5df6-4172-a785-3a326bef95e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.224547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307093dc-5df6-4172-a785-3a326bef95e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.226255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307093dc-5df6-4172-a785-3a326bef95e7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.239911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1007/2020] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:15:47.241088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.251207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214a65b-eb2c-4085-9be1-3504f383cb96 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.260009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc93178-07e1-48c0-93fc-2fbaaf7d2f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.261388 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc93178-07e1-48c0-93fc-2fbaaf7d2f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.261388 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc93178-07e1-48c0-93fc-2fbaaf7d2f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.261874 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc93178-07e1-48c0-93fc-2fbaaf7d2f65 admin admin] PATH_INFO: `/v3/projects/9431c7f47d38408a9431a57532e7468f/users/8a52f860d0484e2689cbfa1cb00b0236/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.264903 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc93178-07e1-48c0-93fc-2fbaaf7d2f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9431c7f47d38408a9431a57532e7468f, user_id=8a52f860d0484e2689cbfa1cb00b0236, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.268336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc93178-07e1-48c0-93fc-2fbaaf7d2f65 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.280306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1014/2021] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:47 2022] PUT /identity/v3/projects/9431c7f47d38408a9431a57532e7468f/users/8a52f860d0484e2689cbfa1cb00b0236/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:47.292276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c2de2-7aad-4ce3-a38d-0a8e741a9c99 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.292568 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c2de2-7aad-4ce3-a38d-0a8e741a9c99 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.292861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c2de2-7aad-4ce3-a38d-0a8e741a9c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.320396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-823c2de2-7aad-4ce3-a38d-0a8e741a9c99 None None] MFA Rules not processed for user `8a52f860d0484e2689cbfa1cb00b0236`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:47.321616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-823c2de2-7aad-4ce3-a38d-0a8e741a9c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:47.331100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:47.390774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc93178-07e1-48c0-93fc-2fbaaf7d2f65 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.392612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8dc93178-07e1-48c0-93fc-2fbaaf7d2f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:47.407375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1008/2022] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:47.407557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:47.420187 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c2de2-7aad-4ce3-a38d-0a8e741a9c99 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.421751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-823c2de2-7aad-4ce3-a38d-0a8e741a9c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:47.466766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc93178-07e1-48c0-93fc-2fbaaf7d2f65 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.472029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.472909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.473186 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.473459 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] PATH_INFO: `/v3/projects/fed0a92759f9474c84c99f6ea30c15d5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.474875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=fed0a92759f9474c84c99f6ea30c15d5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.477399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.479294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c2de2-7aad-4ce3-a38d-0a8e741a9c99 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.483173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbcf897-1730-4040-aabe-fdc9ded9f39a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.484305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcf897-1730-4040-aabe-fdc9ded9f39a service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.484629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcf897-1730-4040-aabe-fdc9ded9f39a service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.484897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbcf897-1730-4040-aabe-fdc9ded9f39a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.495315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcf897-1730-4040-aabe-fdc9ded9f39a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.496775 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbcf897-1730-4040-aabe-fdc9ded9f39a service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.507014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1009/2023] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:47 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:15:47.526230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fed0a92759f9474c84c99f6ea30c15d5', 'request_id': 'req-d7aa0fe0-b411-438d-9b7a-e465618e62ab'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:47.534462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fed0a92759f9474c84c99f6ea30c15d5', 'request_id': 'req-d7aa0fe0-b411-438d-9b7a-e465618e62ab'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:47.541426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fed0a92759f9474c84c99f6ea30c15d5', 'request_id': 'req-d7aa0fe0-b411-438d-9b7a-e465618e62ab'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:47.552618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] The token cache is being invalidate because project fed0a92759f9474c84c99f6ea30c15d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:47.552917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:47.553620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:47.555345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1015/2024] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:47 2022] DELETE /identity/v3/projects/fed0a92759f9474c84c99f6ea30c15d5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:47.560820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7ec3a-c311-4a2c-a9eb-d14cb39dcb92 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.561373 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7ec3a-c311-4a2c-a9eb-d14cb39dcb92 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.561722 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7ec3a-c311-4a2c-a9eb-d14cb39dcb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.581420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6cc7ec3a-c311-4a2c-a9eb-d14cb39dcb92 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:47.582584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6cc7ec3a-c311-4a2c-a9eb-d14cb39dcb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:47.586930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.588442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:47.646017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1010/2025] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:47.654200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa0fe0-b411-438d-9b7a-e465618e62ab admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.657752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a525e8-3d28-4287-8ce0-664d513bf4fa None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.658244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a525e8-3d28-4287-8ce0-664d513bf4fa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.658636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f46c8b-20ad-4555-887d-0f5bc6e2c0e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.658752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a525e8-3d28-4287-8ce0-664d513bf4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.659635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46c8b-20ad-4555-887d-0f5bc6e2c0e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.659967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46c8b-20ad-4555-887d-0f5bc6e2c0e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.660206 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46c8b-20ad-4555-887d-0f5bc6e2c0e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.663617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d1f46c8b-20ad-4555-887d-0f5bc6e2c0e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:47.677423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c3a525e8-3d28-4287-8ce0-664d513bf4fa None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:47.678680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c3a525e8-3d28-4287-8ce0-664d513bf4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:47.687418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1011/2026] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:47.691635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a525e8-3d28-4287-8ce0-664d513bf4fa None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.702777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a525e8-3d28-4287-8ce0-664d513bf4fa None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.707610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc42ff-1358-437b-a68b-58c9611e2da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.708300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc42ff-1358-437b-a68b-58c9611e2da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.708586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc42ff-1358-437b-a68b-58c9611e2da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.708866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc42ff-1358-437b-a68b-58c9611e2da7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.709598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc42ff-1358-437b-a68b-58c9611e2da7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:47.709932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc42ff-1358-437b-a68b-58c9611e2da7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.711574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc42ff-1358-437b-a68b-58c9611e2da7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.727129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1012/2027] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:15:47.728234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc42ff-1358-437b-a68b-58c9611e2da7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.735408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:47.738384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f46c8b-20ad-4555-887d-0f5bc6e2c0e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.739919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f46c8b-20ad-4555-887d-0f5bc6e2c0e6 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.740059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc42ff-1358-437b-a68b-58c9611e2da7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.745193 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b2f90-c6a0-4399-b8aa-4eea41c9c908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.746035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b2f90-c6a0-4399-b8aa-4eea41c9c908 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.746352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b2f90-c6a0-4399-b8aa-4eea41c9c908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.746631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b2f90-c6a0-4399-b8aa-4eea41c9c908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.747402 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b2f90-c6a0-4399-b8aa-4eea41c9c908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.749373 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b2f90-c6a0-4399-b8aa-4eea41c9c908 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.758357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1016/2028] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:47.781021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1013/2029] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:47 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:47.785307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f46c8b-20ad-4555-887d-0f5bc6e2c0e6 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.796820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f46c8b-20ad-4555-887d-0f5bc6e2c0e6 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.857989 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae21e8-bfc5-4424-a8d6-8cbcb1d6d828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.858684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae21e8-bfc5-4424-a8d6-8cbcb1d6d828 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.859010 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae21e8-bfc5-4424-a8d6-8cbcb1d6d828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.859245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae21e8-bfc5-4424-a8d6-8cbcb1d6d828 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.859932 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae21e8-bfc5-4424-a8d6-8cbcb1d6d828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.862549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae21e8-bfc5-4424-a8d6-8cbcb1d6d828 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.895829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1017/2030] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:47 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:47.900162 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b2f90-c6a0-4399-b8aa-4eea41c9c908 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.914904 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b2f90-c6a0-4399-b8aa-4eea41c9c908 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.921385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ce721-f07d-42e5-bb47-645cb4ab2b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.922289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ce721-f07d-42e5-bb47-645cb4ab2b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.922587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ce721-f07d-42e5-bb47-645cb4ab2b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.922890 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ce721-f07d-42e5-bb47-645cb4ab2b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.923929 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ce721-f07d-42e5-bb47-645cb4ab2b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.925800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ce721-f07d-42e5-bb47-645cb4ab2b39 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.928298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:47.942016 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1014/2031] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:47.943874 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae21e8-bfc5-4424-a8d6-8cbcb1d6d828 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.955816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae21e8-bfc5-4424-a8d6-8cbcb1d6d828 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.960506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66334f-19d2-4e09-a536-b3030345b266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.961193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66334f-19d2-4e09-a536-b3030345b266 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:47.961475 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66334f-19d2-4e09-a536-b3030345b266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:47.961749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66334f-19d2-4e09-a536-b3030345b266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:47.962528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66334f-19d2-4e09-a536-b3030345b266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:47.964156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66334f-19d2-4e09-a536-b3030345b266 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:47.978953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1018/2032] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:47.980674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ce721-f07d-42e5-bb47-645cb4ab2b39 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:47.993819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ce721-f07d-42e5-bb47-645cb4ab2b39 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:47.998938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753deaa-1c69-4615-9dda-ea85125a91cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:47.999919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753deaa-1c69-4615-9dda-ea85125a91cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:48.000271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753deaa-1c69-4615-9dda-ea85125a91cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:48.000614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753deaa-1c69-4615-9dda-ea85125a91cd admin admin] PATH_INFO: `/v3/projects/b08648b014714154a3aea018dcc5a874/users/168b2407b60d435a991ddccacf97fb00/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:48.023128 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753deaa-1c69-4615-9dda-ea85125a91cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b08648b014714154a3aea018dcc5a874, user_id=168b2407b60d435a991ddccacf97fb00, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:48.026507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753deaa-1c69-4615-9dda-ea85125a91cd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:48.060098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1015/2033] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:47 2022] PUT /identity/v3/projects/b08648b014714154a3aea018dcc5a874/users/168b2407b60d435a991ddccacf97fb00/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:48.063013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93c699-37f8-4004-8cdf-2bf6aabf5875 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:48.063013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93c699-37f8-4004-8cdf-2bf6aabf5875 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:48.063224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93c699-37f8-4004-8cdf-2bf6aabf5875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:48.100513 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8a93c699-37f8-4004-8cdf-2bf6aabf5875 None None] MFA Rules not processed for user `168b2407b60d435a991ddccacf97fb00`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:48.101842 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8a93c699-37f8-4004-8cdf-2bf6aabf5875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:48.182532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1019/2034] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:48.187174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753deaa-1c69-4615-9dda-ea85125a91cd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:48.267520 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753deaa-1c69-4615-9dda-ea85125a91cd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:48.272273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb67edf-f55e-45b8-a12a-d3f8a04eb202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:48.272943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb67edf-f55e-45b8-a12a-d3f8a04eb202 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:48.273224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb67edf-f55e-45b8-a12a-d3f8a04eb202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:48.273494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb67edf-f55e-45b8-a12a-d3f8a04eb202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:48.274227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb67edf-f55e-45b8-a12a-d3f8a04eb202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:48.276126 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb67edf-f55e-45b8-a12a-d3f8a04eb202 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:48.303532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1016/2035] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:48 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:48.308037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93c699-37f8-4004-8cdf-2bf6aabf5875 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:48.319893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93c699-37f8-4004-8cdf-2bf6aabf5875 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:48.385772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494c02a-780e-4fcb-8530-d39c1431bc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:48.386505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494c02a-780e-4fcb-8530-d39c1431bc95 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:48.386795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494c02a-780e-4fcb-8530-d39c1431bc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:48.387081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494c02a-780e-4fcb-8530-d39c1431bc95 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:48.387786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494c02a-780e-4fcb-8530-d39c1431bc95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:48.389731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7494c02a-780e-4fcb-8530-d39c1431bc95 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:48.398549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:48.411838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1020/2036] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:48 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:48.416172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb67edf-f55e-45b8-a12a-d3f8a04eb202 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:48.440870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb67edf-f55e-45b8-a12a-d3f8a04eb202 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:48.454117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f26b7-fa43-4f42-a2c2-496c086cca26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:48.454946 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f26b7-fa43-4f42-a2c2-496c086cca26 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:48.455415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f26b7-fa43-4f42-a2c2-496c086cca26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:48.455695 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f26b7-fa43-4f42-a2c2-496c086cca26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:48.456656 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f26b7-fa43-4f42-a2c2-496c086cca26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:48.458412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f26b7-fa43-4f42-a2c2-496c086cca26 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:48.477427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1017/2037] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:48.479164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494c02a-780e-4fcb-8530-d39c1431bc95 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:48.489695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494c02a-780e-4fcb-8530-d39c1431bc95 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:48.494838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0787c-0bef-4552-9c48-854c4a624e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:48.495737 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0787c-0bef-4552-9c48-854c4a624e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:48.496023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0787c-0bef-4552-9c48-854c4a624e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:48.496301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0787c-0bef-4552-9c48-854c4a624e58 admin admin] PATH_INFO: `/v3/projects/d4203d6c9b0144acb93b468620095f61/users/9934a14ec74d47cfb35b1a7193986601/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:48.514885 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0787c-0bef-4552-9c48-854c4a624e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4203d6c9b0144acb93b468620095f61, user_id=9934a14ec74d47cfb35b1a7193986601, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:48.517930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0787c-0bef-4552-9c48-854c4a624e58 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:48.535635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1021/2038] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:48 2022] PUT /identity/v3/projects/d4203d6c9b0144acb93b468620095f61/users/9934a14ec74d47cfb35b1a7193986601/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:48.537386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f26b7-fa43-4f42-a2c2-496c086cca26 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:48.548496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f26b7-fa43-4f42-a2c2-496c086cca26 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:48.621597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036095d-e6a7-4804-b0f3-341d36b5ce9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:48.622337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036095d-e6a7-4804-b0f3-341d36b5ce9b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:48.622624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036095d-e6a7-4804-b0f3-341d36b5ce9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:48.622902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036095d-e6a7-4804-b0f3-341d36b5ce9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:48.623650 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036095d-e6a7-4804-b0f3-341d36b5ce9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:48.625434 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036095d-e6a7-4804-b0f3-341d36b5ce9b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:48.640069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1018/2039] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:48.644462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0787c-0bef-4552-9c48-854c4a624e58 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:48.655871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0787c-0bef-4552-9c48-854c4a624e58 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:48.661471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf6291-d4e5-4eed-82aa-fa9b7f3b955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:48.662426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf6291-d4e5-4eed-82aa-fa9b7f3b955c admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:48.662720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf6291-d4e5-4eed-82aa-fa9b7f3b955c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:48.663002 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf6291-d4e5-4eed-82aa-fa9b7f3b955c admin admin] PATH_INFO: `/v3/projects/d4203d6c9b0144acb93b468620095f61/users/9934a14ec74d47cfb35b1a7193986601/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:48.665608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf6291-d4e5-4eed-82aa-fa9b7f3b955c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4203d6c9b0144acb93b468620095f61, user_id=9934a14ec74d47cfb35b1a7193986601, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:48.668331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf6291-d4e5-4eed-82aa-fa9b7f3b955c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:48.687533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1022/2040] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:48 2022] PUT /identity/v3/projects/d4203d6c9b0144acb93b468620095f61/users/9934a14ec74d47cfb35b1a7193986601/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:48.694592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30112b-d12f-49e1-980c-fc45088ee573 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:48.694918 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30112b-d12f-49e1-980c-fc45088ee573 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:48.695206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30112b-d12f-49e1-980c-fc45088ee573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:48.734776 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-af30112b-d12f-49e1-980c-fc45088ee573 None None] MFA Rules not processed for user `9934a14ec74d47cfb35b1a7193986601`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:48.736733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-af30112b-d12f-49e1-980c-fc45088ee573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:48.792490 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:48.857451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf6291-d4e5-4eed-82aa-fa9b7f3b955c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:48.859243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a3cf6291-d4e5-4eed-82aa-fa9b7f3b955c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:48.874295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1019/2041] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:48.890862 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30112b-d12f-49e1-980c-fc45088ee573 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:48.958315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf6291-d4e5-4eed-82aa-fa9b7f3b955c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:48.968063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:48.969284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:48.969675 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:48.970082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] PATH_INFO: `/v3/users/22d4a830c43f477ebd93beb22711d267` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:48.970642 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30112b-d12f-49e1-980c-fc45088ee573 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:48.972169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22d4a830c43f477ebd93beb22711d267)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:48.975780 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:48.979192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0d15c-1ea1-4f22-8dcd-945ecac571a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:48.979879 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0d15c-1ea1-4f22-8dcd-945ecac571a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:48.980164 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0d15c-1ea1-4f22-8dcd-945ecac571a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:48.980440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0d15c-1ea1-4f22-8dcd-945ecac571a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:48.992015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0d15c-1ea1-4f22-8dcd-945ecac571a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:48.993492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0d15c-1ea1-4f22-8dcd-945ecac571a7 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:49.008805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1020/2042] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:49.068683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22d4a830c43f477ebd93beb22711d267'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:49.068683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22d4a830c43f477ebd93beb22711d267'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:49.087798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22d4a830c43f477ebd93beb22711d267'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:49.102632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22d4a830c43f477ebd93beb22711d267'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:49.108410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22d4a830c43f477ebd93beb22711d267'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:49.143045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1023/2043] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:48 2022] DELETE /identity/v3/users/22d4a830c43f477ebd93beb22711d267 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:49.147333 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feccdd6-9279-4da4-acae-3b50f9101841 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:49.147626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feccdd6-9279-4da4-acae-3b50f9101841 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:49.147905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feccdd6-9279-4da4-acae-3b50f9101841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:49.183371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8feccdd6-9279-4da4-acae-3b50f9101841 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:49.185039 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8feccdd6-9279-4da4-acae-3b50f9101841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:49.311886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1021/2044] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:49.345848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:49.347550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:49.477320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:49.507843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65916235-be96-40c9-a114-ea58661f6fb2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:49.514090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627ea0-bc3d-4d51-8d5d-f0def202b96f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:49.514839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627ea0-bc3d-4d51-8d5d-f0def202b96f service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:49.515132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627ea0-bc3d-4d51-8d5d-f0def202b96f service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:49.515408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627ea0-bc3d-4d51-8d5d-f0def202b96f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:49.534006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db627ea0-bc3d-4d51-8d5d-f0def202b96f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:49.537542 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db627ea0-bc3d-4d51-8d5d-f0def202b96f service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:49.555977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1024/2045] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:50.078876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feccdd6-9279-4da4-acae-3b50f9101841 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:50.081469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8feccdd6-9279-4da4-acae-3b50f9101841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:50.109792 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feccdd6-9279-4da4-acae-3b50f9101841 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:50.115255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:50.116137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:50.116416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:50.116690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] PATH_INFO: `/v3/projects/d11acdd77f9840f984b16bd64b67eeea` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:50.119473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11acdd77f9840f984b16bd64b67eeea)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:50.122422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:50.155993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627ea0-bc3d-4d51-8d5d-f0def202b96f service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:50.157330 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-db627ea0-bc3d-4d51-8d5d-f0def202b96f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:50.175712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627ea0-bc3d-4d51-8d5d-f0def202b96f service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:50.187671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a566c8-c759-4120-b0be-7254c287010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:50.189122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a566c8-c759-4120-b0be-7254c287010c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:50.189717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a566c8-c759-4120-b0be-7254c287010c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:50.190211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a566c8-c759-4120-b0be-7254c287010c admin admin] PATH_INFO: `/v3/users/168b2407b60d435a991ddccacf97fb00` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:50.192091 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a566c8-c759-4120-b0be-7254c287010c admin admin] RBAC: Authorizing `identity:delete_user(user_id=168b2407b60d435a991ddccacf97fb00)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:50.194532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a566c8-c759-4120-b0be-7254c287010c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:50.220343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11acdd77f9840f984b16bd64b67eeea', 'request_id': 'req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.231897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11acdd77f9840f984b16bd64b67eeea', 'request_id': 'req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.240790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11acdd77f9840f984b16bd64b67eeea', 'request_id': 'req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.247077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:50.262649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03a566c8-c759-4120-b0be-7254c287010c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '168b2407b60d435a991ddccacf97fb00'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.263386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03a566c8-c759-4120-b0be-7254c287010c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '168b2407b60d435a991ddccacf97fb00'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.266617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] The token cache is being invalidate because project d11acdd77f9840f984b16bd64b67eeea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:50.267683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.268764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.270487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1022/2046] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:50 2022] DELETE /identity/v3/projects/d11acdd77f9840f984b16bd64b67eeea => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:50.272015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03a566c8-c759-4120-b0be-7254c287010c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '168b2407b60d435a991ddccacf97fb00'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.274843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:50.276254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:50.279822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03a566c8-c759-4120-b0be-7254c287010c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '168b2407b60d435a991ddccacf97fb00'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.280512 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03a566c8-c759-4120-b0be-7254c287010c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '168b2407b60d435a991ddccacf97fb00'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.303900 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1025/2047] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:50 2022] DELETE /identity/v3/users/168b2407b60d435a991ddccacf97fb00 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:50.309011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e49e-0849-4409-bfe6-efda2844581d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:50.309305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e49e-0849-4409-bfe6-efda2844581d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:50.309587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e49e-0849-4409-bfe6-efda2844581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:50.340657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-6c55e49e-0849-4409-bfe6-efda2844581d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:50.341923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6c55e49e-0849-4409-bfe6-efda2844581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:50.381922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dce623-ba9d-48b5-b3df-e8f09fe8cc40 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:50.420584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1026/2048] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:50.431514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e49e-0849-4409-bfe6-efda2844581d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:50.433004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6c55e49e-0849-4409-bfe6-efda2844581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:50.453070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:50.454207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:50.455018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:50.455341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] PATH_INFO: `/v3/users/839e75237c9346f68385de18406a831e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:50.457129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] RBAC: Authorizing `identity:delete_user(user_id=839e75237c9346f68385de18406a831e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:50.459095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:50.498680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e49e-0849-4409-bfe6-efda2844581d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:50.503958 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85059bcb-3b3c-483a-adff-2fa683f6f668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:50.504959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85059bcb-3b3c-483a-adff-2fa683f6f668 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:50.505307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85059bcb-3b3c-483a-adff-2fa683f6f668 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:50.505654 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85059bcb-3b3c-483a-adff-2fa683f6f668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:50.514064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:50.521328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85059bcb-3b3c-483a-adff-2fa683f6f668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:50.524396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85059bcb-3b3c-483a-adff-2fa683f6f668 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:50.525468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '839e75237c9346f68385de18406a831e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.526252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '839e75237c9346f68385de18406a831e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.526867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '839e75237c9346f68385de18406a831e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.538480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '839e75237c9346f68385de18406a831e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.540544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1027/2049] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:15:50.551800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '839e75237c9346f68385de18406a831e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:50.575498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1023/2050] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:50 2022] DELETE /identity/v3/users/839e75237c9346f68385de18406a831e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:51.106575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85059bcb-3b3c-483a-adff-2fa683f6f668 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:51.108493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-85059bcb-3b3c-483a-adff-2fa683f6f668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:51.146323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85059bcb-3b3c-483a-adff-2fa683f6f668 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:51.171205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:51.173276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:51.227916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:51.254021 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:51.255352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:51.255779 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:51.256729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] PATH_INFO: `/v3/projects/b08648b014714154a3aea018dcc5a874` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:51.258243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b08648b014714154a3aea018dcc5a874)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:51.260444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:51.283089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fafa4-fe4a-4e7e-9a9b-7e1a224ee68a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:51.291141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:51.292118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:51.292440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:51.292720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] PATH_INFO: `/v3/projects/4ad76210190e4c8f91c17d2369b0b8f0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:51.295225 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad76210190e4c8f91c17d2369b0b8f0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:51.297318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:51.348380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b08648b014714154a3aea018dcc5a874', 'request_id': 'req-b63a81f7-a796-4292-a72f-9c833b8fdb63'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.362536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b08648b014714154a3aea018dcc5a874', 'request_id': 'req-b63a81f7-a796-4292-a72f-9c833b8fdb63'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.372289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b08648b014714154a3aea018dcc5a874', 'request_id': 'req-b63a81f7-a796-4292-a72f-9c833b8fdb63'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.372549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad76210190e4c8f91c17d2369b0b8f0', 'request_id': 'req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.385455 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad76210190e4c8f91c17d2369b0b8f0', 'request_id': 'req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.388287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] The token cache is being invalidate because project b08648b014714154a3aea018dcc5a874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:51.388578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.389402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.391276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1028/2051] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:51 2022] DELETE /identity/v3/projects/b08648b014714154a3aea018dcc5a874 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:51.394755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad76210190e4c8f91c17d2369b0b8f0', 'request_id': 'req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.396289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:51.398415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:51.410671 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] The token cache is being invalidate because project 4ad76210190e4c8f91c17d2369b0b8f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:51.411308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.412337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.414630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1024/2052] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:51 2022] DELETE /identity/v3/projects/4ad76210190e4c8f91c17d2369b0b8f0 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:51.418587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:51.420201 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:51.478838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a81f7-a796-4292-a72f-9c833b8fdb63 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:51.497315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8cb7d-84be-4410-9c90-2db4da6c11f4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:51.504044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:51.505133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:51.508376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:51.508376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] PATH_INFO: `/v3/users/a06c04db53d0464ba79fba5cb56c0496` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:51.508376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a06c04db53d0464ba79fba5cb56c0496)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:51.510931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:51.513529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:51.553862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb6568-5eb0-41d5-acd6-f2869fbc098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:51.554858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb6568-5eb0-41d5-acd6-f2869fbc098b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:51.555184 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb6568-5eb0-41d5-acd6-f2869fbc098b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:51.555460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb6568-5eb0-41d5-acd6-f2869fbc098b admin admin] PATH_INFO: `/v3/users/9934a14ec74d47cfb35b1a7193986601` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:51.556679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb6568-5eb0-41d5-acd6-f2869fbc098b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9934a14ec74d47cfb35b1a7193986601)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:51.558663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb6568-5eb0-41d5-acd6-f2869fbc098b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:51.567691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a06c04db53d0464ba79fba5cb56c0496'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.568629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a06c04db53d0464ba79fba5cb56c0496'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.569357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a06c04db53d0464ba79fba5cb56c0496'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.582539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a06c04db53d0464ba79fba5cb56c0496'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.584896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:51.595430 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a06c04db53d0464ba79fba5cb56c0496'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.612006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-71bb6568-5eb0-41d5-acd6-f2869fbc098b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9934a14ec74d47cfb35b1a7193986601'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.612682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-71bb6568-5eb0-41d5-acd6-f2869fbc098b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9934a14ec74d47cfb35b1a7193986601'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.618705 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1025/2053] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:51 2022] DELETE /identity/v3/users/a06c04db53d0464ba79fba5cb56c0496 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:51.619566 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:51.621405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:51.630731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-71bb6568-5eb0-41d5-acd6-f2869fbc098b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9934a14ec74d47cfb35b1a7193986601'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.639244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-71bb6568-5eb0-41d5-acd6-f2869fbc098b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9934a14ec74d47cfb35b1a7193986601'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.640053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-71bb6568-5eb0-41d5-acd6-f2869fbc098b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9934a14ec74d47cfb35b1a7193986601'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.665519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1029/2054] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:51 2022] DELETE /identity/v3/users/9934a14ec74d47cfb35b1a7193986601 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:51.691830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eada07-49df-4494-b2a9-b7681d418b07 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:51.760534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:51.761465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:51.761815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:51.762206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] PATH_INFO: `/v3/users/23943474dccc41b6bdd23d164d093278` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:51.763513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23943474dccc41b6bdd23d164d093278)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:51.765414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:51.864887 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23943474dccc41b6bdd23d164d093278'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.865772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23943474dccc41b6bdd23d164d093278'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.866622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23943474dccc41b6bdd23d164d093278'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.890993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23943474dccc41b6bdd23d164d093278'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.901081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23943474dccc41b6bdd23d164d093278'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:51.923233 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:51.934968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1026/2055] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:51 2022] DELETE /identity/v3/users/23943474dccc41b6bdd23d164d093278 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:51.936655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28710a94-2c2d-4e96-a9fc-c3f961f91d8a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:51.937084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28710a94-2c2d-4e96-a9fc-c3f961f91d8a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:51.938282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28710a94-2c2d-4e96-a9fc-c3f961f91d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:51.967052 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-28710a94-2c2d-4e96-a9fc-c3f961f91d8a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:51.968807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-28710a94-2c2d-4e96-a9fc-c3f961f91d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.025301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:52.030449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.070352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1030/2056] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:15:52.090618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28710a94-2c2d-4e96-a9fc-c3f961f91d8a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:52.092508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-28710a94-2c2d-4e96-a9fc-c3f961f91d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.153511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e2903-fc0c-4efa-a2b3-2d7875df2813 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:52.159524 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:52.160421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.160795 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.161390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] PATH_INFO: `/v3/projects/1c4c0e33dccf4b20b5de9807afdc6535` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.162613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c4c0e33dccf4b20b5de9807afdc6535)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:52.164581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:52.187785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28710a94-2c2d-4e96-a9fc-c3f961f91d8a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:52.192575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77444dac-5960-47c0-add1-2bf0adefd042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:52.193392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77444dac-5960-47c0-add1-2bf0adefd042 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.193519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77444dac-5960-47c0-add1-2bf0adefd042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.193797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77444dac-5960-47c0-add1-2bf0adefd042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.201532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:52.206781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77444dac-5960-47c0-add1-2bf0adefd042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:52.209623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77444dac-5960-47c0-add1-2bf0adefd042 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:52.227028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1031/2057] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:52.228279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77444dac-5960-47c0-add1-2bf0adefd042 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:52.228860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c4c0e33dccf4b20b5de9807afdc6535', 'request_id': 'req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.229997 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-77444dac-5960-47c0-add1-2bf0adefd042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.239957 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c4c0e33dccf4b20b5de9807afdc6535', 'request_id': 'req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.250779 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77444dac-5960-47c0-add1-2bf0adefd042 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:52.290054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c4c0e33dccf4b20b5de9807afdc6535', 'request_id': 'req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.313163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] The token cache is being invalidate because project 1c4c0e33dccf4b20b5de9807afdc6535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:52.313464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.314309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-aef76e8d-b1cd-49f6-a9f5-9a0f2a80dd91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.316882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1027/2058] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:52 2022] DELETE /identity/v3/projects/1c4c0e33dccf4b20b5de9807afdc6535 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:52.322717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a79fb-5926-4b92-819b-60e3bf1b3598 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.323111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a79fb-5926-4b92-819b-60e3bf1b3598 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.324544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a79fb-5926-4b92-819b-60e3bf1b3598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.351621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8a6a79fb-5926-4b92-819b-60e3bf1b3598 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:52.353345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8a6a79fb-5926-4b92-819b-60e3bf1b3598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.374847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7cbde-4947-441b-a9fa-6363861b7ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:52.376200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7cbde-4947-441b-a9fa-6363861b7ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.376405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7cbde-4947-441b-a9fa-6363861b7ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.377020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7cbde-4947-441b-a9fa-6363861b7ad0 admin admin] PATH_INFO: `/v3/users/185b640f6d034bdeaee4d053b145eecf` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.379328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7cbde-4947-441b-a9fa-6363861b7ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=185b640f6d034bdeaee4d053b145eecf)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:52.381527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7cbde-4947-441b-a9fa-6363861b7ad0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:52.443165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1028/2059] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:52.446808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a79fb-5926-4b92-819b-60e3bf1b3598 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:52.448918 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8a6a79fb-5926-4b92-819b-60e3bf1b3598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.458754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-31f7cbde-4947-441b-a9fa-6363861b7ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '185b640f6d034bdeaee4d053b145eecf'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.459447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-31f7cbde-4947-441b-a9fa-6363861b7ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '185b640f6d034bdeaee4d053b145eecf'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.466712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a79fb-5926-4b92-819b-60e3bf1b3598 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:52.471440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-31f7cbde-4947-441b-a9fa-6363861b7ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '185b640f6d034bdeaee4d053b145eecf'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.473901 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:52.474872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.475232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.475582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] PATH_INFO: `/v3/projects/d4203d6c9b0144acb93b468620095f61` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.476866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4203d6c9b0144acb93b468620095f61)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:52.479274 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:52.480088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-31f7cbde-4947-441b-a9fa-6363861b7ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '185b640f6d034bdeaee4d053b145eecf'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.480876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-31f7cbde-4947-441b-a9fa-6363861b7ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '185b640f6d034bdeaee4d053b145eecf'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.502147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1032/2060] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:52 2022] DELETE /identity/v3/users/185b640f6d034bdeaee4d053b145eecf => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:52.503569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91451b3d-0152-4f94-a4a6-444159c49369 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.503927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91451b3d-0152-4f94-a4a6-444159c49369 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.504282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91451b3d-0152-4f94-a4a6-444159c49369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.514420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:52.524994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-91451b3d-0152-4f94-a4a6-444159c49369 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:52.526614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-91451b3d-0152-4f94-a4a6-444159c49369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.550228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4203d6c9b0144acb93b468620095f61', 'request_id': 'req-c834973a-1793-4777-b2d6-7a95bba8a5f2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.564231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4203d6c9b0144acb93b468620095f61', 'request_id': 'req-c834973a-1793-4777-b2d6-7a95bba8a5f2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.571755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4203d6c9b0144acb93b468620095f61', 'request_id': 'req-c834973a-1793-4777-b2d6-7a95bba8a5f2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.585237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] The token cache is being invalidate because project d4203d6c9b0144acb93b468620095f61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:52.585536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.588680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c834973a-1793-4777-b2d6-7a95bba8a5f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.589958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1029/2061] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:52 2022] DELETE /identity/v3/projects/d4203d6c9b0144acb93b468620095f61 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:52.592404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e805d6-e50c-4ade-a2b8-2ed10594928e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.592946 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e805d6-e50c-4ade-a2b8-2ed10594928e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.593458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e805d6-e50c-4ade-a2b8-2ed10594928e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.613952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-80e805d6-e50c-4ade-a2b8-2ed10594928e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:52.615319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-80e805d6-e50c-4ade-a2b8-2ed10594928e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.618072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1033/2062] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:52.620081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa360c2-8172-4855-8a66-bcedc910436c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.620566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa360c2-8172-4855-8a66-bcedc910436c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.621069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa360c2-8172-4855-8a66-bcedc910436c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.643185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-6fa360c2-8172-4855-8a66-bcedc910436c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:52.644108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6fa360c2-8172-4855-8a66-bcedc910436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.652521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:52.681502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1030/2063] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:52.682611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e805d6-e50c-4ade-a2b8-2ed10594928e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:52.684099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-80e805d6-e50c-4ade-a2b8-2ed10594928e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.696963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e805d6-e50c-4ade-a2b8-2ed10594928e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:52.702360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:52.703304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.703592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.703867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] PATH_INFO: `/v3/projects/1079706b2e4848b192e1b674433d36cc` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.705117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1079706b2e4848b192e1b674433d36cc)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:52.707045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:52.721238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1034/2064] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:52.721608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa360c2-8172-4855-8a66-bcedc910436c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:52.732367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa360c2-8172-4855-8a66-bcedc910436c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:52.738773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091159be-9301-4676-9e88-4981f02d643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:52.739424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091159be-9301-4676-9e88-4981f02d643e admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.739714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091159be-9301-4676-9e88-4981f02d643e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.740029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091159be-9301-4676-9e88-4981f02d643e admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.740804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091159be-9301-4676-9e88-4981f02d643e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:52.741113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091159be-9301-4676-9e88-4981f02d643e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:52.742767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091159be-9301-4676-9e88-4981f02d643e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:52.757082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1035/2065] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:52.758069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091159be-9301-4676-9e88-4981f02d643e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:52.760093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-091159be-9301-4676-9e88-4981f02d643e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.761933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1079706b2e4848b192e1b674433d36cc', 'request_id': 'req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.770862 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1079706b2e4848b192e1b674433d36cc', 'request_id': 'req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.777819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1079706b2e4848b192e1b674433d36cc', 'request_id': 'req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.793261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] The token cache is being invalidate because project 1079706b2e4848b192e1b674433d36cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:52.793546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.794365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-adb22fa6-81ae-4bb8-bc75-bbda80fbd623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:52.796002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1031/2066] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:52 2022] DELETE /identity/v3/projects/1079706b2e4848b192e1b674433d36cc => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:52.797264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0aa0ef-5ba7-4207-8335-257da6c8b403 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.797553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0aa0ef-5ba7-4207-8335-257da6c8b403 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.797828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0aa0ef-5ba7-4207-8335-257da6c8b403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.816871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8c0aa0ef-5ba7-4207-8335-257da6c8b403 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:52.818149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8c0aa0ef-5ba7-4207-8335-257da6c8b403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.823108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091159be-9301-4676-9e88-4981f02d643e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:52.851125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:52.873225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83897dd9-6e40-45a0-b0d3-bfac2bbfdc28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:52.873914 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83897dd9-6e40-45a0-b0d3-bfac2bbfdc28 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.874224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83897dd9-6e40-45a0-b0d3-bfac2bbfdc28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.874507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83897dd9-6e40-45a0-b0d3-bfac2bbfdc28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.876351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-83897dd9-6e40-45a0-b0d3-bfac2bbfdc28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.885388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1032/2067] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:52.886235 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0aa0ef-5ba7-4207-8335-257da6c8b403 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:52.887464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8c0aa0ef-5ba7-4207-8335-257da6c8b403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.899917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0aa0ef-5ba7-4207-8335-257da6c8b403 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:52.907784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0447a4-a26a-4e73-8ebd-010e471b3e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:52.908529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0447a4-a26a-4e73-8ebd-010e471b3e2f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:52.908801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0447a4-a26a-4e73-8ebd-010e471b3e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:52.909057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0447a4-a26a-4e73-8ebd-010e471b3e2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:52.909764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0447a4-a26a-4e73-8ebd-010e471b3e2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:52.911762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0447a4-a26a-4e73-8ebd-010e471b3e2f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:52.944008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1033/2068] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:52 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:52.944008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0447a4-a26a-4e73-8ebd-010e471b3e2f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:52.945723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7e0447a4-a26a-4e73-8ebd-010e471b3e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:52.949397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83897dd9-6e40-45a0-b0d3-bfac2bbfdc28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:52.950972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83897dd9-6e40-45a0-b0d3-bfac2bbfdc28 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:52.962618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1036/2069] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:52.963885 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83897dd9-6e40-45a0-b0d3-bfac2bbfdc28 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:52.974186 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83897dd9-6e40-45a0-b0d3-bfac2bbfdc28 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:52.978318 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:53.033432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0447a4-a26a-4e73-8ebd-010e471b3e2f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.038623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.039523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.039855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.040130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] PATH_INFO: `/v3/users/8a52f860d0484e2689cbfa1cb00b0236` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.041367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a52f860d0484e2689cbfa1cb00b0236)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.043000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a1431-6fe5-461a-8d63-cf8db15c94bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.043497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.043866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1431-6fe5-461a-8d63-cf8db15c94bd admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.044185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1431-6fe5-461a-8d63-cf8db15c94bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.044447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a1431-6fe5-461a-8d63-cf8db15c94bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.045158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a1431-6fe5-461a-8d63-cf8db15c94bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:53.045462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a1431-6fe5-461a-8d63-cf8db15c94bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.047134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a1431-6fe5-461a-8d63-cf8db15c94bd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.064242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1037/2070] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:53.065009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a1431-6fe5-461a-8d63-cf8db15c94bd admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.076421 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a1431-6fe5-461a-8d63-cf8db15c94bd admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.082598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86758a-b392-4146-8f2c-520a6f7029ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.083281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86758a-b392-4146-8f2c-520a6f7029ef admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.083569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86758a-b392-4146-8f2c-520a6f7029ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.083850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86758a-b392-4146-8f2c-520a6f7029ef admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.084552 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86758a-b392-4146-8f2c-520a6f7029ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.086534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86758a-b392-4146-8f2c-520a6f7029ef admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.110853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a52f860d0484e2689cbfa1cb00b0236'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.111863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a52f860d0484e2689cbfa1cb00b0236'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.112929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1038/2071] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:53 2022] POST /identity/v3/users => generated 292 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:53.113384 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a52f860d0484e2689cbfa1cb00b0236'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.116779 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86758a-b392-4146-8f2c-520a6f7029ef admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.119686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ad86758a-b392-4146-8f2c-520a6f7029ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:53.146977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86758a-b392-4146-8f2c-520a6f7029ef admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.152835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a52f860d0484e2689cbfa1cb00b0236'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.161438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fbd59-ede5-46c3-a27a-2d8017c10fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.162731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fbd59-ede5-46c3-a27a-2d8017c10fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.163022 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fbd59-ede5-46c3-a27a-2d8017c10fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.163287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fbd59-ede5-46c3-a27a-2d8017c10fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.164003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fbd59-ede5-46c3-a27a-2d8017c10fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.166377 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fbd59-ede5-46c3-a27a-2d8017c10fb0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.167281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a52f860d0484e2689cbfa1cb00b0236'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.199733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1034/2072] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:52 2022] DELETE /identity/v3/users/8a52f860d0484e2689cbfa1cb00b0236 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:53.200503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.201994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:53.208125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1039/2073] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:53 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:53.212256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fbd59-ede5-46c3-a27a-2d8017c10fb0 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.244419 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:53.279911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa83acf-af93-4be7-860f-449d0fc565df admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.299258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fbd59-ede5-46c3-a27a-2d8017c10fb0 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.305579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aef9a-c75b-4df7-b639-9358a36a2a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.306491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aef9a-c75b-4df7-b639-9358a36a2a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.306875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aef9a-c75b-4df7-b639-9358a36a2a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.308387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aef9a-c75b-4df7-b639-9358a36a2a16 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.308387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4aef9a-c75b-4df7-b639-9358a36a2a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.310449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4aef9a-c75b-4df7-b639-9358a36a2a16 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.356976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1040/2074] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:53 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:53.357086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99e29f-63c2-4c40-836c-8361d1a1522c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.357086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99e29f-63c2-4c40-836c-8361d1a1522c admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.357086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99e29f-63c2-4c40-836c-8361d1a1522c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.357418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99e29f-63c2-4c40-836c-8361d1a1522c admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.358382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99e29f-63c2-4c40-836c-8361d1a1522c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.359130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aef9a-c75b-4df7-b639-9358a36a2a16 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.360540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99e29f-63c2-4c40-836c-8361d1a1522c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.379287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aef9a-c75b-4df7-b639-9358a36a2a16 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.389302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc8603-f9f9-4232-a0a5-2b2e82ebd549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.390121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc8603-f9f9-4232-a0a5-2b2e82ebd549 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.390490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc8603-f9f9-4232-a0a5-2b2e82ebd549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.390825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc8603-f9f9-4232-a0a5-2b2e82ebd549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.395909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc8603-f9f9-4232-a0a5-2b2e82ebd549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.396260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1035/2075] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:53.396361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc8603-f9f9-4232-a0a5-2b2e82ebd549 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.397710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99e29f-63c2-4c40-836c-8361d1a1522c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.407647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1041/2076] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:53.411464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc8603-f9f9-4232-a0a5-2b2e82ebd549 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.418413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99e29f-63c2-4c40-836c-8361d1a1522c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.427332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0306d-aa39-4384-ace3-512e12cc0dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.428166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0306d-aa39-4384-ace3-512e12cc0dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.428505 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0306d-aa39-4384-ace3-512e12cc0dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.429416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0306d-aa39-4384-ace3-512e12cc0dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.433771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc8603-f9f9-4232-a0a5-2b2e82ebd549 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.433989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0306d-aa39-4384-ace3-512e12cc0dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.433989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0306d-aa39-4384-ace3-512e12cc0dd6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.450047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a83bd-192c-4565-b1d6-995385db1c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.450752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a83bd-192c-4565-b1d6-995385db1c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.451040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a83bd-192c-4565-b1d6-995385db1c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.452223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1036/2077] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:53.452353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a83bd-192c-4565-b1d6-995385db1c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.453087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a83bd-192c-4565-b1d6-995385db1c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.453360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0306d-aa39-4384-ace3-512e12cc0dd6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.454807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a83bd-192c-4565-b1d6-995385db1c76 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.456072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2fb0306d-aa39-4384-ace3-512e12cc0dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:53.475538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0306d-aa39-4384-ace3-512e12cc0dd6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.479696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1042/2078] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:15:53.480566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a83bd-192c-4565-b1d6-995385db1c76 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.481326 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.482348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.482703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.483108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] PATH_INFO: `/v3/projects/49293146ea2f4fbd9c457a821f567991` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.484503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49293146ea2f4fbd9c457a821f567991)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.487220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.499965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a83bd-192c-4565-b1d6-995385db1c76 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.507930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722886db-055d-4c66-bb07-60661282a134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.509374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722886db-055d-4c66-bb07-60661282a134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.509715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722886db-055d-4c66-bb07-60661282a134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.510066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722886db-055d-4c66-bb07-60661282a134 admin admin] PATH_INFO: `/v3/projects/88b5d59895d441ffb0122673f9131bad/users/721fcdf3811d4d37ab1304ba1972ff5f/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.530840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722886db-055d-4c66-bb07-60661282a134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b5d59895d441ffb0122673f9131bad, user_id=721fcdf3811d4d37ab1304ba1972ff5f, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.534099 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722886db-055d-4c66-bb07-60661282a134 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.560050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1043/2079] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:53 2022] PUT /identity/v3/projects/88b5d59895d441ffb0122673f9131bad/users/721fcdf3811d4d37ab1304ba1972ff5f/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:53.561357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722886db-055d-4c66-bb07-60661282a134 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.578947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:53.583332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49293146ea2f4fbd9c457a821f567991', 'request_id': 'req-66052d6d-7b18-497f-af78-93ac877997b1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.595111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49293146ea2f4fbd9c457a821f567991', 'request_id': 'req-66052d6d-7b18-497f-af78-93ac877997b1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.612627 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49293146ea2f4fbd9c457a821f567991', 'request_id': 'req-66052d6d-7b18-497f-af78-93ac877997b1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.628589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] The token cache is being invalidate because project 49293146ea2f4fbd9c457a821f567991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:53.628977 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.629729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-66052d6d-7b18-497f-af78-93ac877997b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.631791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1037/2080] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:53 2022] DELETE /identity/v3/projects/49293146ea2f4fbd9c457a821f567991 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:53.633510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec38cf6-9444-4228-af10-b1702a9ca60c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.636998 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec38cf6-9444-4228-af10-b1702a9ca60c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.636998 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec38cf6-9444-4228-af10-b1702a9ca60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.657550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722886db-055d-4c66-bb07-60661282a134 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.672495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-dec38cf6-9444-4228-af10-b1702a9ca60c None None] MFA Rules not processed for user `721fcdf3811d4d37ab1304ba1972ff5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:53.673781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dec38cf6-9444-4228-af10-b1702a9ca60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:53.721138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:53.735255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c0edd-0e88-4e3b-af9f-28a478ad147f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.736290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c0edd-0e88-4e3b-af9f-28a478ad147f admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.736682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c0edd-0e88-4e3b-af9f-28a478ad147f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.737214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c0edd-0e88-4e3b-af9f-28a478ad147f admin admin] PATH_INFO: `/v3/projects/bab6831ffd9e4097b440e55aadf9bef8/users/409a822e17d84649be4ba569b2cb3e2e/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.751502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1038/2081] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:53.752708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c0edd-0e88-4e3b-af9f-28a478ad147f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab6831ffd9e4097b440e55aadf9bef8, user_id=409a822e17d84649be4ba569b2cb3e2e, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.752926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a5e2b-790a-48fc-9f29-1e0bc24a6da8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.753218 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a5e2b-790a-48fc-9f29-1e0bc24a6da8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.753492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a5e2b-790a-48fc-9f29-1e0bc24a6da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.756097 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c0edd-0e88-4e3b-af9f-28a478ad147f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.773719 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1044/2082] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:53 2022] PUT /identity/v3/projects/bab6831ffd9e4097b440e55aadf9bef8/users/409a822e17d84649be4ba569b2cb3e2e/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:53.774605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c0edd-0e88-4e3b-af9f-28a478ad147f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.776132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ee2c0edd-0e88-4e3b-af9f-28a478ad147f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:53.776268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1e6a5e2b-790a-48fc-9f29-1e0bc24a6da8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:53.777360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1e6a5e2b-790a-48fc-9f29-1e0bc24a6da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:53.845703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c0edd-0e88-4e3b-af9f-28a478ad147f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.845962 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1039/2083] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:53.847227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a5e2b-790a-48fc-9f29-1e0bc24a6da8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:53.848567 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1e6a5e2b-790a-48fc-9f29-1e0bc24a6da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:53.850841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.851718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.851984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.852255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] PATH_INFO: `/v3/projects/9431c7f47d38408a9431a57532e7468f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.853436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9431c7f47d38408a9431a57532e7468f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.855451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.861993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a5e2b-790a-48fc-9f29-1e0bc24a6da8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:53.866872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb7bf1-f95b-4818-8958-54c7ea7bf962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:53.867555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb7bf1-f95b-4818-8958-54c7ea7bf962 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.867843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb7bf1-f95b-4818-8958-54c7ea7bf962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.868124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb7bf1-f95b-4818-8958-54c7ea7bf962 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.868838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb7bf1-f95b-4818-8958-54c7ea7bf962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:53.870813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb7bf1-f95b-4818-8958-54c7ea7bf962 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:53.900540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1040/2084] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:53 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:53.902297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075787f-6699-432f-a8db-3673cd120287 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.902720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075787f-6699-432f-a8db-3673cd120287 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.903128 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075787f-6699-432f-a8db-3673cd120287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.906888 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:53.919217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9431c7f47d38408a9431a57532e7468f', 'request_id': 'req-db24c6c9-001d-45c7-b87c-f32878a59e86'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.927653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9431c7f47d38408a9431a57532e7468f', 'request_id': 'req-db24c6c9-001d-45c7-b87c-f32878a59e86'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.930516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d075787f-6699-432f-a8db-3673cd120287 None None] MFA Rules not processed for user `409a822e17d84649be4ba569b2cb3e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:53.931785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d075787f-6699-432f-a8db-3673cd120287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:53.934222 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9431c7f47d38408a9431a57532e7468f', 'request_id': 'req-db24c6c9-001d-45c7-b87c-f32878a59e86'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.948867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] The token cache is being invalidate because project 9431c7f47d38408a9431a57532e7468f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:53.949284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.950582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-db24c6c9-001d-45c7-b87c-f32878a59e86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:53.953501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1045/2085] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:53 2022] DELETE /identity/v3/projects/9431c7f47d38408a9431a57532e7468f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:53.956197 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2439e2-019c-4a56-817f-9f5cbe6c4afc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:53.956798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2439e2-019c-4a56-817f-9f5cbe6c4afc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:53.957180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2439e2-019c-4a56-817f-9f5cbe6c4afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:53.976264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-fe2439e2-019c-4a56-817f-9f5cbe6c4afc None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:53.977621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fe2439e2-019c-4a56-817f-9f5cbe6c4afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:54.017109 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1041/2086] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:54.020622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075787f-6699-432f-a8db-3673cd120287 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.021631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d075787f-6699-432f-a8db-3673cd120287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:54.043736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1046/2087] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:54.048778 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500d05-d365-419f-81f1-cd3283a92981 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.049443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500d05-d365-419f-81f1-cd3283a92981 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.049925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500d05-d365-419f-81f1-cd3283a92981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.072542 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-19500d05-d365-419f-81f1-cd3283a92981 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:54.073805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-19500d05-d365-419f-81f1-cd3283a92981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:54.074420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:54.083463 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1047/2088] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:54.084260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19500d05-d365-419f-81f1-cd3283a92981 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.085567 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-19500d05-d365-419f-81f1-cd3283a92981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:54.097990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075787f-6699-432f-a8db-3673cd120287 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.098385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19500d05-d365-419f-81f1-cd3283a92981 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.102879 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c1183-c2f6-425d-a76c-a2e077812494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.103023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f95ee6-10cf-4a6a-b083-d5534f83236f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.103557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c1183-c2f6-425d-a76c-a2e077812494 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.103666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f95ee6-10cf-4a6a-b083-d5534f83236f admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.103836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c1183-c2f6-425d-a76c-a2e077812494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.103946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f95ee6-10cf-4a6a-b083-d5534f83236f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.104115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c1183-c2f6-425d-a76c-a2e077812494 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.104268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f95ee6-10cf-4a6a-b083-d5534f83236f admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.104815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c1183-c2f6-425d-a76c-a2e077812494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.104920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f95ee6-10cf-4a6a-b083-d5534f83236f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.106804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f95ee6-10cf-4a6a-b083-d5534f83236f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.106911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c1183-c2f6-425d-a76c-a2e077812494 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.129643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1042/2089] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:54.130537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c1183-c2f6-425d-a76c-a2e077812494 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.140813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1048/2090] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:54.142400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab800e1-698c-4692-9bfc-e19c5aa93d74 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.142762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab800e1-698c-4692-9bfc-e19c5aa93d74 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.143043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab800e1-698c-4692-9bfc-e19c5aa93d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.146591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c1183-c2f6-425d-a76c-a2e077812494 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.166665 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-eab800e1-698c-4692-9bfc-e19c5aa93d74 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:54.168190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-eab800e1-698c-4692-9bfc-e19c5aa93d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:54.215526 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f3632-18f1-4221-8548-ef204b6cbd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.216217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f3632-18f1-4221-8548-ef204b6cbd41 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.216496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f3632-18f1-4221-8548-ef204b6cbd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.216786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f3632-18f1-4221-8548-ef204b6cbd41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.217547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f3632-18f1-4221-8548-ef204b6cbd41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:54.217856 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f3632-18f1-4221-8548-ef204b6cbd41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.219490 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f3632-18f1-4221-8548-ef204b6cbd41 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.231809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1043/2091] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:54.232757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f3632-18f1-4221-8548-ef204b6cbd41 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.244959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1049/2092] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:54.245381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab800e1-698c-4692-9bfc-e19c5aa93d74 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.249776 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f3632-18f1-4221-8548-ef204b6cbd41 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.255236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab800e1-698c-4692-9bfc-e19c5aa93d74 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.255465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35007fb5-ec5e-4524-b510-a585d6cda7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.256178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35007fb5-ec5e-4524-b510-a585d6cda7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.256466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35007fb5-ec5e-4524-b510-a585d6cda7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.256756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35007fb5-ec5e-4524-b510-a585d6cda7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.257502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35007fb5-ec5e-4524-b510-a585d6cda7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.259803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35007fb5-ec5e-4524-b510-a585d6cda7ff admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.260341 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e15b6-2dab-4558-a2f0-cc74c5c968d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.261009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e15b6-2dab-4558-a2f0-cc74c5c968d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.261294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e15b6-2dab-4558-a2f0-cc74c5c968d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.261569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e15b6-2dab-4558-a2f0-cc74c5c968d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.262309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e15b6-2dab-4558-a2f0-cc74c5c968d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.264200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e15b6-2dab-4558-a2f0-cc74c5c968d0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.271639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1044/2093] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:54.272429 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35007fb5-ec5e-4524-b510-a585d6cda7ff admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.283221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35007fb5-ec5e-4524-b510-a585d6cda7ff admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.291936 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f6b01-df76-4364-90e8-c226b026096e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.291936 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6b01-df76-4364-90e8-c226b026096e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.292168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6b01-df76-4364-90e8-c226b026096e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.292398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6b01-df76-4364-90e8-c226b026096e admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.292712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1050/2094] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/users => generated 288 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:54.293104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f6b01-df76-4364-90e8-c226b026096e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.293917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e15b6-2dab-4558-a2f0-cc74c5c968d0 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.295245 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f6b01-df76-4364-90e8-c226b026096e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.305123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e15b6-2dab-4558-a2f0-cc74c5c968d0 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.310228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12b433-2893-49e9-963c-358122c0a2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.311105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12b433-2893-49e9-963c-358122c0a2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.311846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12b433-2893-49e9-963c-358122c0a2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.312121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12b433-2893-49e9-963c-358122c0a2d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.312893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12b433-2893-49e9-963c-358122c0a2d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:54.313526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12b433-2893-49e9-963c-358122c0a2d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.315700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12b433-2893-49e9-963c-358122c0a2d3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.327846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1045/2095] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:54.327846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f6b01-df76-4364-90e8-c226b026096e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.336303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1051/2096] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:54.336303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12b433-2893-49e9-963c-358122c0a2d3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.346546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12b433-2893-49e9-963c-358122c0a2d3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.350604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:54.394009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f6b01-df76-4364-90e8-c226b026096e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.399511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29fb71-6f26-4e37-a92b-018aa02eab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.400226 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29fb71-6f26-4e37-a92b-018aa02eab80 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.400509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29fb71-6f26-4e37-a92b-018aa02eab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.400768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29fb71-6f26-4e37-a92b-018aa02eab80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.401525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29fb71-6f26-4e37-a92b-018aa02eab80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.403174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29fb71-6f26-4e37-a92b-018aa02eab80 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.410754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec1dc6-b13e-4f8f-b06a-0369fac88152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.411431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec1dc6-b13e-4f8f-b06a-0369fac88152 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.411714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec1dc6-b13e-4f8f-b06a-0369fac88152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.411993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec1dc6-b13e-4f8f-b06a-0369fac88152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.412724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec1dc6-b13e-4f8f-b06a-0369fac88152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.413830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1046/2097] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:54.414357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec1dc6-b13e-4f8f-b06a-0369fac88152 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.414643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29fb71-6f26-4e37-a92b-018aa02eab80 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.425183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29fb71-6f26-4e37-a92b-018aa02eab80 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.425512 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1052/2098] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:54.426384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec1dc6-b13e-4f8f-b06a-0369fac88152 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.430496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b3954-aa38-460f-abb0-c0c482187cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.431155 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b3954-aa38-460f-abb0-c0c482187cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.431443 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b3954-aa38-460f-abb0-c0c482187cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.431698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b3954-aa38-460f-abb0-c0c482187cb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.432395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b3954-aa38-460f-abb0-c0c482187cb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.434375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b3954-aa38-460f-abb0-c0c482187cb8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.437597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec1dc6-b13e-4f8f-b06a-0369fac88152 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.441457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbe746-1b22-4736-a3f0-be6c5a088fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.442360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbe746-1b22-4736-a3f0-be6c5a088fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.442640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbe746-1b22-4736-a3f0-be6c5a088fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.443355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbe746-1b22-4736-a3f0-be6c5a088fd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.444430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbe746-1b22-4736-a3f0-be6c5a088fd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.446703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbe746-1b22-4736-a3f0-be6c5a088fd8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.461793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1047/2099] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/users => generated 281 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:54.461793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b3954-aa38-460f-abb0-c0c482187cb8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.475677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b3954-aa38-460f-abb0-c0c482187cb8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.478942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1053/2100] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:54.479930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbe746-1b22-4736-a3f0-be6c5a088fd8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.481943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbcdf3-7375-48bb-90f7-81e9aef3c38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.483099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbcdf3-7375-48bb-90f7-81e9aef3c38b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.483511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbcdf3-7375-48bb-90f7-81e9aef3c38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.484021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbcdf3-7375-48bb-90f7-81e9aef3c38b admin admin] PATH_INFO: `/v3/projects/f184d0a0a46b405898c17059cdb26a9b/users/ca0c6d6bd5c243dc8f181de43ef1e05e/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.497548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:54.501976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbcdf3-7375-48bb-90f7-81e9aef3c38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f184d0a0a46b405898c17059cdb26a9b, user_id=ca0c6d6bd5c243dc8f181de43ef1e05e, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.505576 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbcdf3-7375-48bb-90f7-81e9aef3c38b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.520923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1048/2101] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:54 2022] PUT /identity/v3/projects/f184d0a0a46b405898c17059cdb26a9b/users/ca0c6d6bd5c243dc8f181de43ef1e05e/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:54.524540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbcdf3-7375-48bb-90f7-81e9aef3c38b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.544107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbe746-1b22-4736-a3f0-be6c5a088fd8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.593648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbcdf3-7375-48bb-90f7-81e9aef3c38b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.599886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c00b5-a2c2-4037-b3d6-f07a6048c7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.600840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c00b5-a2c2-4037-b3d6-f07a6048c7ef admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.601190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c00b5-a2c2-4037-b3d6-f07a6048c7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.601902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c00b5-a2c2-4037-b3d6-f07a6048c7ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.602780 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c00b5-a2c2-4037-b3d6-f07a6048c7ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.604448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c00b5-a2c2-4037-b3d6-f07a6048c7ef admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.617932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1049/2102] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:54.617932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c00b5-a2c2-4037-b3d6-f07a6048c7ef admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.619270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0c0f0-9589-41fd-94ab-cd93f0631ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.619982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0c0f0-9589-41fd-94ab-cd93f0631ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.620434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0c0f0-9589-41fd-94ab-cd93f0631ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.620714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0c0f0-9589-41fd-94ab-cd93f0631ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.621502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b0c0f0-9589-41fd-94ab-cd93f0631ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.623209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b0c0f0-9589-41fd-94ab-cd93f0631ce0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.627109 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c00b5-a2c2-4037-b3d6-f07a6048c7ef admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.632252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95345d8-9f0a-4ff4-8db3-d5360641a398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.632930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95345d8-9f0a-4ff4-8db3-d5360641a398 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.633212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95345d8-9f0a-4ff4-8db3-d5360641a398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.633488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95345d8-9f0a-4ff4-8db3-d5360641a398 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.634566 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95345d8-9f0a-4ff4-8db3-d5360641a398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.635052 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1054/2103] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:54.636457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eabe89-ac49-4999-940c-c8cf0c45acb0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.636764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eabe89-ac49-4999-940c-c8cf0c45acb0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.637044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eabe89-ac49-4999-940c-c8cf0c45acb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.639727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95345d8-9f0a-4ff4-8db3-d5360641a398 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.667526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-66eabe89-ac49-4999-940c-c8cf0c45acb0 None None] MFA Rules not processed for user `ca0c6d6bd5c243dc8f181de43ef1e05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:54.668727 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-66eabe89-ac49-4999-940c-c8cf0c45acb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:54.669480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1050/2104] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/users => generated 284 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:54.670497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95345d8-9f0a-4ff4-8db3-d5360641a398 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.682631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95345d8-9f0a-4ff4-8db3-d5360641a398 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.684637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:54.692717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e90945-9a77-4df6-a97c-0b4f11ba5ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.694363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e90945-9a77-4df6-a97c-0b4f11ba5ade admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.694817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e90945-9a77-4df6-a97c-0b4f11ba5ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.695233 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e90945-9a77-4df6-a97c-0b4f11ba5ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.696121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e90945-9a77-4df6-a97c-0b4f11ba5ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.698696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e90945-9a77-4df6-a97c-0b4f11ba5ade admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.713949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1051/2105] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:54.713949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e90945-9a77-4df6-a97c-0b4f11ba5ade admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.722226 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e90945-9a77-4df6-a97c-0b4f11ba5ade admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.732698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8556c-7fd5-43a9-adf6-3bbc7f786002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.733681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8556c-7fd5-43a9-adf6-3bbc7f786002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.734024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8556c-7fd5-43a9-adf6-3bbc7f786002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.734306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8556c-7fd5-43a9-adf6-3bbc7f786002 admin admin] PATH_INFO: `/v3/projects/c2cc190d6c8b45ab84e797235dead675/users/42de3812651a4bb893ff3ffae706718e/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.751261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1055/2106] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:54.751261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eabe89-ac49-4999-940c-c8cf0c45acb0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.753781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8556c-7fd5-43a9-adf6-3bbc7f786002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2cc190d6c8b45ab84e797235dead675, user_id=42de3812651a4bb893ff3ffae706718e, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.756644 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8556c-7fd5-43a9-adf6-3bbc7f786002 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.763631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eabe89-ac49-4999-940c-c8cf0c45acb0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.768590 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12521287-581e-4b47-9c29-bc5d47c62b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.769258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12521287-581e-4b47-9c29-bc5d47c62b7c admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.769550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12521287-581e-4b47-9c29-bc5d47c62b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.769850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12521287-581e-4b47-9c29-bc5d47c62b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.770622 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12521287-581e-4b47-9c29-bc5d47c62b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.772227 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12521287-581e-4b47-9c29-bc5d47c62b7c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.772565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1052/2107] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:54 2022] PUT /identity/v3/projects/c2cc190d6c8b45ab84e797235dead675/users/42de3812651a4bb893ff3ffae706718e/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:54.782401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8556c-7fd5-43a9-adf6-3bbc7f786002 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.785718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1056/2108] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:54.786502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12521287-581e-4b47-9c29-bc5d47c62b7c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.796172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12521287-581e-4b47-9c29-bc5d47c62b7c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.846143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8556c-7fd5-43a9-adf6-3bbc7f786002 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.852043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36d023-4324-4355-99b7-75651ca60d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.852885 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36d023-4324-4355-99b7-75651ca60d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.853168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36d023-4324-4355-99b7-75651ca60d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.853448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36d023-4324-4355-99b7-75651ca60d41 admin admin] PATH_INFO: `/v3/projects/52c4fbe8c2dd4de3936143e9afe1727c/users/ad03d0839de2455d955d0336a8f53023/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.855919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce43d5a-6b05-4898-9336-8d8389125805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.857405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce43d5a-6b05-4898-9336-8d8389125805 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.857405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce43d5a-6b05-4898-9336-8d8389125805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.857405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce43d5a-6b05-4898-9336-8d8389125805 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.857925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce43d5a-6b05-4898-9336-8d8389125805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.859866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce43d5a-6b05-4898-9336-8d8389125805 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.870177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36d023-4324-4355-99b7-75651ca60d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c4fbe8c2dd4de3936143e9afe1727c, user_id=ad03d0839de2455d955d0336a8f53023, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.873140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36d023-4324-4355-99b7-75651ca60d41 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.888372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1053/2109] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:54 2022] PUT /identity/v3/projects/52c4fbe8c2dd4de3936143e9afe1727c/users/ad03d0839de2455d955d0336a8f53023/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:54.891935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1057/2110] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:54.892067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f26d5a-ee23-41de-99f7-86636b955f58 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.892067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f26d5a-ee23-41de-99f7-86636b955f58 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.892300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce43d5a-6b05-4898-9336-8d8389125805 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.892393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f26d5a-ee23-41de-99f7-86636b955f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.922317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c7f26d5a-ee23-41de-99f7-86636b955f58 None None] MFA Rules not processed for user `42de3812651a4bb893ff3ffae706718e`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:54.923592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c7f26d5a-ee23-41de-99f7-86636b955f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:54.928659 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:54.962303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce43d5a-6b05-4898-9336-8d8389125805 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:54.968184 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a9009-ac86-4743-ae20-1b09b200e56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:54.968863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a9009-ac86-4743-ae20-1b09b200e56f admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.969162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a9009-ac86-4743-ae20-1b09b200e56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.969431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a9009-ac86-4743-ae20-1b09b200e56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.970204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a9009-ac86-4743-ae20-1b09b200e56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:54.972045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a9009-ac86-4743-ae20-1b09b200e56f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:54.985270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1058/2111] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:54.985270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a9009-ac86-4743-ae20-1b09b200e56f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:54.990966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1054/2112] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:54.992393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8f516-3dd5-4c2a-aecf-acd024259dd4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:54.992756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8f516-3dd5-4c2a-aecf-acd024259dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:54.993102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8f516-3dd5-4c2a-aecf-acd024259dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:54.994899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a9009-ac86-4743-ae20-1b09b200e56f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.000243 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bc15a-f8f6-4f7b-8116-c72684cb0744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.000405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bc15a-f8f6-4f7b-8116-c72684cb0744 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.000638 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bc15a-f8f6-4f7b-8116-c72684cb0744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.000914 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bc15a-f8f6-4f7b-8116-c72684cb0744 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.001667 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bc15a-f8f6-4f7b-8116-c72684cb0744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.003645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bc15a-f8f6-4f7b-8116-c72684cb0744 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.021404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-46e8f516-3dd5-4c2a-aecf-acd024259dd4 None None] MFA Rules not processed for user `ad03d0839de2455d955d0336a8f53023`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:55.022655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-46e8f516-3dd5-4c2a-aecf-acd024259dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:55.024536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1059/2113] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/users => generated 299 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:55.025329 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bc15a-f8f6-4f7b-8116-c72684cb0744 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.041640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bc15a-f8f6-4f7b-8116-c72684cb0744 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.046419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e573d-3a07-48a4-9c85-004e43b6437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.047311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e573d-3a07-48a4-9c85-004e43b6437f admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.047596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e573d-3a07-48a4-9c85-004e43b6437f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.047884 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e573d-3a07-48a4-9c85-004e43b6437f admin admin] PATH_INFO: `/v3/projects/4ab8e41403bb426bb54d265fa98bdbb2/users/3fa9bb26cf6546419c4e2a64ca44fba6/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.050344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:55.064796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e573d-3a07-48a4-9c85-004e43b6437f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab8e41403bb426bb54d265fa98bdbb2, user_id=3fa9bb26cf6546419c4e2a64ca44fba6, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.067722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e573d-3a07-48a4-9c85-004e43b6437f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.083039 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1060/2114] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:55 2022] PUT /identity/v3/projects/4ab8e41403bb426bb54d265fa98bdbb2/users/3fa9bb26cf6546419c4e2a64ca44fba6/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:55.083881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e573d-3a07-48a4-9c85-004e43b6437f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.103726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1055/2115] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:55.105113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8f516-3dd5-4c2a-aecf-acd024259dd4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.125177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8f516-3dd5-4c2a-aecf-acd024259dd4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.164083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e573d-3a07-48a4-9c85-004e43b6437f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.168935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ebc28-a6b0-4449-a91f-d0d349c16a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.169602 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ebc28-a6b0-4449-a91f-d0d349c16a6a admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.169910 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ebc28-a6b0-4449-a91f-d0d349c16a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.170201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ebc28-a6b0-4449-a91f-d0d349c16a6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.170901 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ebc28-a6b0-4449-a91f-d0d349c16a6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.172856 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ebc28-a6b0-4449-a91f-d0d349c16a6a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.189313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f61b3-4104-47bb-bf0c-540580cf12bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.190176 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f61b3-4104-47bb-bf0c-540580cf12bf admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.190485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f61b3-4104-47bb-bf0c-540580cf12bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.190870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f61b3-4104-47bb-bf0c-540580cf12bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.191657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f61b3-4104-47bb-bf0c-540580cf12bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.193585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f61b3-4104-47bb-bf0c-540580cf12bf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.199897 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1061/2116] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:55 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:55.201506 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb57857-a0c6-4910-89df-1561e9c15327 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.201923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb57857-a0c6-4910-89df-1561e9c15327 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.202371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb57857-a0c6-4910-89df-1561e9c15327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.204495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1056/2117] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:55.205326 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f61b3-4104-47bb-bf0c-540580cf12bf admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.207006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-496f61b3-4104-47bb-bf0c-540580cf12bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:55.231427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1bb57857-a0c6-4910-89df-1561e9c15327 None None] MFA Rules not processed for user `3fa9bb26cf6546419c4e2a64ca44fba6`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:55.233291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1bb57857-a0c6-4910-89df-1561e9c15327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:55.254346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:55.286194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f61b3-4104-47bb-bf0c-540580cf12bf admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.318238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1062/2118] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:55.319132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb57857-a0c6-4910-89df-1561e9c15327 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.340658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51eebbb-7bde-4e97-9d06-7a0a62069584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.341314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51eebbb-7bde-4e97-9d06-7a0a62069584 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.341597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51eebbb-7bde-4e97-9d06-7a0a62069584 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.341873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51eebbb-7bde-4e97-9d06-7a0a62069584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.377984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:55.382898 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb57857-a0c6-4910-89df-1561e9c15327 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.389428 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e42d3-b3a0-4792-8721-70fff8a9bbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.390204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e42d3-b3a0-4792-8721-70fff8a9bbb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.390574 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e42d3-b3a0-4792-8721-70fff8a9bbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.390915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e42d3-b3a0-4792-8721-70fff8a9bbb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.391676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e42d3-b3a0-4792-8721-70fff8a9bbb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.393912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e42d3-b3a0-4792-8721-70fff8a9bbb9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.404017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51eebbb-7bde-4e97-9d06-7a0a62069584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.405740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51eebbb-7bde-4e97-9d06-7a0a62069584 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.418208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1063/2119] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:55 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:55.418208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e42d3-b3a0-4792-8721-70fff8a9bbb9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.420524 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1057/2120] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:55.421205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51eebbb-7bde-4e97-9d06-7a0a62069584 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.426084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e42d3-b3a0-4792-8721-70fff8a9bbb9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.431126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c1bd1-320d-48e3-844f-4a37cb343625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.431988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c1bd1-320d-48e3-844f-4a37cb343625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.432300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c1bd1-320d-48e3-844f-4a37cb343625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.432876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c1bd1-320d-48e3-844f-4a37cb343625 admin admin] PATH_INFO: `/v3/projects/e8358c6201134e52a0ab36146cb470b7/users/f694f84f0e464d3bb8fd3ce3ca4d2c8c/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.435540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51eebbb-7bde-4e97-9d06-7a0a62069584 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.440915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c04950-4f1c-432f-995b-2dbbc4aa46d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.441574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04950-4f1c-432f-995b-2dbbc4aa46d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.441856 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04950-4f1c-432f-995b-2dbbc4aa46d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.442174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04950-4f1c-432f-995b-2dbbc4aa46d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.442872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c04950-4f1c-432f-995b-2dbbc4aa46d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.444770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c04950-4f1c-432f-995b-2dbbc4aa46d8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.448349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c1bd1-320d-48e3-844f-4a37cb343625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8358c6201134e52a0ab36146cb470b7, user_id=f694f84f0e464d3bb8fd3ce3ca4d2c8c, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.451046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c1bd1-320d-48e3-844f-4a37cb343625 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.465853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1064/2121] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:55 2022] PUT /identity/v3/projects/e8358c6201134e52a0ab36146cb470b7/users/f694f84f0e464d3bb8fd3ce3ca4d2c8c/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:55.466695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c1bd1-320d-48e3-844f-4a37cb343625 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.473968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1058/2122] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:55 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:55.474877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c04950-4f1c-432f-995b-2dbbc4aa46d8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.532824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c1bd1-320d-48e3-844f-4a37cb343625 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.538426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fe963-5f0d-4eb3-b713-9cd8f808da25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.539102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fe963-5f0d-4eb3-b713-9cd8f808da25 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.539382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fe963-5f0d-4eb3-b713-9cd8f808da25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.539629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fe963-5f0d-4eb3-b713-9cd8f808da25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.540384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fe963-5f0d-4eb3-b713-9cd8f808da25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.542031 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fe963-5f0d-4eb3-b713-9cd8f808da25 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.546068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c04950-4f1c-432f-995b-2dbbc4aa46d8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.551004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.551882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.552166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.552442 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] PATH_INFO: `/v3/users/ad03d0839de2455d955d0336a8f53023` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.553647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad03d0839de2455d955d0336a8f53023)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.555638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.555914 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1065/2123] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:55.556677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fe963-5f0d-4eb3-b713-9cd8f808da25 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.566328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fe963-5f0d-4eb3-b713-9cd8f808da25 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.570939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297315a-1317-458f-98d4-5dd7b510097b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.571610 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297315a-1317-458f-98d4-5dd7b510097b admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.571890 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297315a-1317-458f-98d4-5dd7b510097b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.572170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297315a-1317-458f-98d4-5dd7b510097b admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.572884 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297315a-1317-458f-98d4-5dd7b510097b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.574540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297315a-1317-458f-98d4-5dd7b510097b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.580906 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:55.584978 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1066/2124] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:55.586192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297315a-1317-458f-98d4-5dd7b510097b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.596413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297315a-1317-458f-98d4-5dd7b510097b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.601442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4221d4e-43d4-402c-9498-e1a79792cd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.602185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4221d4e-43d4-402c-9498-e1a79792cd75 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.602469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4221d4e-43d4-402c-9498-e1a79792cd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.602744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4221d4e-43d4-402c-9498-e1a79792cd75 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.603431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4221d4e-43d4-402c-9498-e1a79792cd75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.605355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4221d4e-43d4-402c-9498-e1a79792cd75 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.609230 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad03d0839de2455d955d0336a8f53023'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:55.609928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad03d0839de2455d955d0336a8f53023'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:55.610575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad03d0839de2455d955d0336a8f53023'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:55.620433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad03d0839de2455d955d0336a8f53023'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:55.626755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1067/2125] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:55 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:55.627571 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4221d4e-43d4-402c-9498-e1a79792cd75 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.627878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad03d0839de2455d955d0336a8f53023'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:55.629337 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d4221d4e-43d4-402c-9498-e1a79792cd75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:55.641380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4221d4e-43d4-402c-9498-e1a79792cd75 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.646237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1059/2126] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:15:55 2022] DELETE /identity/v3/users/ad03d0839de2455d955d0336a8f53023 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:55.646377 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6b417-fcb9-4cd6-b984-06b3f1c97004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.647018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.647203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6b417-fcb9-4cd6-b984-06b3f1c97004 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.647476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6b417-fcb9-4cd6-b984-06b3f1c97004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.647754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6b417-fcb9-4cd6-b984-06b3f1c97004 admin admin] PATH_INFO: `/v3/projects/41f4066e2e4941c482307c56b59bd7bc/users/8dad2caf1fac48b5b67e248030036e52/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.648380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:55.663617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6b417-fcb9-4cd6-b984-06b3f1c97004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f4066e2e4941c482307c56b59bd7bc, user_id=8dad2caf1fac48b5b67e248030036e52, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.666367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6b417-fcb9-4cd6-b984-06b3f1c97004 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.680160 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1068/2127] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:55 2022] PUT /identity/v3/projects/41f4066e2e4941c482307c56b59bd7bc/users/8dad2caf1fac48b5b67e248030036e52/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:55.683411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6b417-fcb9-4cd6-b984-06b3f1c97004 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.683411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d0f6b417-fcb9-4cd6-b984-06b3f1c97004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:55.715231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55631661-b76c-4f1f-b0bb-eef9eca58549 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.746239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6b417-fcb9-4cd6-b984-06b3f1c97004 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.751053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e371726-4d77-433d-b7f4-879e255b2a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.751826 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371726-4d77-433d-b7f4-879e255b2a3e admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.752275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371726-4d77-433d-b7f4-879e255b2a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.752921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371726-4d77-433d-b7f4-879e255b2a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.753998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e371726-4d77-433d-b7f4-879e255b2a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.755613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e371726-4d77-433d-b7f4-879e255b2a3e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.766657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1069/2128] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:55.768323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4a53b-0c9b-4d3e-be33-5897940b34a0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.768714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4a53b-0c9b-4d3e-be33-5897940b34a0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.768999 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4a53b-0c9b-4d3e-be33-5897940b34a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.777602 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:55.784947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a7fe4-9220-4085-a5c2-db1b354d4e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.786055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a7fe4-9220-4085-a5c2-db1b354d4e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.786574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a7fe4-9220-4085-a5c2-db1b354d4e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.786691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a7fe4-9220-4085-a5c2-db1b354d4e3b admin admin] PATH_INFO: `/v3/projects/e8358c6201134e52a0ab36146cb470b7/users/f694f84f0e464d3bb8fd3ce3ca4d2c8c/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.789046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a7fe4-9220-4085-a5c2-db1b354d4e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8358c6201134e52a0ab36146cb470b7, user_id=f694f84f0e464d3bb8fd3ce3ca4d2c8c, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.790186 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1ac4a53b-0c9b-4d3e-be33-5897940b34a0 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:55.791459 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1ac4a53b-0c9b-4d3e-be33-5897940b34a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:55.792102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a7fe4-9220-4085-a5c2-db1b354d4e3b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.799719 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1070/2129] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:55.800545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4a53b-0c9b-4d3e-be33-5897940b34a0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.806939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1060/2130] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:55 2022] PUT /identity/v3/projects/e8358c6201134e52a0ab36146cb470b7/users/f694f84f0e464d3bb8fd3ce3ca4d2c8c/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:55.807825 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a7fe4-9220-4085-a5c2-db1b354d4e3b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.819284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4a53b-0c9b-4d3e-be33-5897940b34a0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.857449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:55.879156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a7fe4-9220-4085-a5c2-db1b354d4e3b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:55.883860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738d916-0c87-402e-b2b0-cef0e4dac0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.884685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69ee62-67fb-4ea9-a177-30f3885e66ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:55.884879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738d916-0c87-402e-b2b0-cef0e4dac0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.885164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738d916-0c87-402e-b2b0-cef0e4dac0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.885433 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738d916-0c87-402e-b2b0-cef0e4dac0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.885953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69ee62-67fb-4ea9-a177-30f3885e66ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.886199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738d916-0c87-402e-b2b0-cef0e4dac0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.886303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69ee62-67fb-4ea9-a177-30f3885e66ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.886630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69ee62-67fb-4ea9-a177-30f3885e66ce admin admin] PATH_INFO: `/v3/projects/7d1b32acf10e41978fcb731da8270425/users/6bc22e7f4fe74030abdb84f4eee10d5c/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.887830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738d916-0c87-402e-b2b0-cef0e4dac0e8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.899260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1071/2131] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:55.900309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738d916-0c87-402e-b2b0-cef0e4dac0e8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:55.902007 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1738d916-0c87-402e-b2b0-cef0e4dac0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:55.907157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69ee62-67fb-4ea9-a177-30f3885e66ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1b32acf10e41978fcb731da8270425, user_id=6bc22e7f4fe74030abdb84f4eee10d5c, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:55.910101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69ee62-67fb-4ea9-a177-30f3885e66ce admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:55.925488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1061/2132] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:55 2022] PUT /identity/v3/projects/7d1b32acf10e41978fcb731da8270425/users/6bc22e7f4fe74030abdb84f4eee10d5c/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:55.927040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4aab8-cdbd-4f4c-a926-03a4f357e36d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:55.927352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4aab8-cdbd-4f4c-a926-03a4f357e36d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:55.927629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4aab8-cdbd-4f4c-a926-03a4f357e36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:55.955916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-16b4aab8-cdbd-4f4c-a926-03a4f357e36d None None] MFA Rules not processed for user `f694f84f0e464d3bb8fd3ce3ca4d2c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:55.957564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-16b4aab8-cdbd-4f4c-a926-03a4f357e36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:55.975323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738d916-0c87-402e-b2b0-cef0e4dac0e8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:56.022911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf217c-96ac-49ff-b80f-bb440ffbb2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:56.023581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf217c-96ac-49ff-b80f-bb440ffbb2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.023872 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf217c-96ac-49ff-b80f-bb440ffbb2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.024150 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf217c-96ac-49ff-b80f-bb440ffbb2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.027294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1062/2133] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:56.028110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4aab8-cdbd-4f4c-a926-03a4f357e36d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:56.084497 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf217c-96ac-49ff-b80f-bb440ffbb2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:56.086054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf217c-96ac-49ff-b80f-bb440ffbb2e5 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:56.089942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:56.090843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4aab8-cdbd-4f4c-a926-03a4f357e36d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:56.095659 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49540666-4b65-4856-b014-3178df1074bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:56.096589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49540666-4b65-4856-b014-3178df1074bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.096752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1072/2134] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:56.097633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf217c-96ac-49ff-b80f-bb440ffbb2e5 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:56.097967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49540666-4b65-4856-b014-3178df1074bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.098471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49540666-4b65-4856-b014-3178df1074bf admin admin] PATH_INFO: `/v3/projects/41f4066e2e4941c482307c56b59bd7bc/users/8dad2caf1fac48b5b67e248030036e52/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.100621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49540666-4b65-4856-b014-3178df1074bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f4066e2e4941c482307c56b59bd7bc, user_id=8dad2caf1fac48b5b67e248030036e52, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:56.104368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49540666-4b65-4856-b014-3178df1074bf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:56.106864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf217c-96ac-49ff-b80f-bb440ffbb2e5 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:56.112740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40849f74-a0e0-4fc8-a640-fed3ac2033fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:56.113431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40849f74-a0e0-4fc8-a640-fed3ac2033fc admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.113666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40849f74-a0e0-4fc8-a640-fed3ac2033fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.114032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40849f74-a0e0-4fc8-a640-fed3ac2033fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.114808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40849f74-a0e0-4fc8-a640-fed3ac2033fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:56.116306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40849f74-a0e0-4fc8-a640-fed3ac2033fc admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:56.125449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1063/2135] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:56 2022] PUT /identity/v3/projects/41f4066e2e4941c482307c56b59bd7bc/users/8dad2caf1fac48b5b67e248030036e52/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:56.125449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49540666-4b65-4856-b014-3178df1074bf admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:56.130870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1073/2136] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:15:56.130870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b65c6-4930-4e2d-84f6-d139ece3032f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.130870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b65c6-4930-4e2d-84f6-d139ece3032f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.130870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b65c6-4930-4e2d-84f6-d139ece3032f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.131297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-49540666-4b65-4856-b014-3178df1074bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:56.160238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-db2b65c6-4930-4e2d-84f6-d139ece3032f None None] MFA Rules not processed for user `8dad2caf1fac48b5b67e248030036e52`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:56.161784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-db2b65c6-4930-4e2d-84f6-d139ece3032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:56.201270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:56.214780 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49540666-4b65-4856-b014-3178df1074bf admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:56.218771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5c70c-2090-45a1-8b2f-9f568b22d5a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:56.219551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5c70c-2090-45a1-8b2f-9f568b22d5a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.219837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5c70c-2090-45a1-8b2f-9f568b22d5a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.220113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5c70c-2090-45a1-8b2f-9f568b22d5a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.222350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-24e5c70c-2090-45a1-8b2f-9f568b22d5a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:56.251347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1074/2137] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:56.252712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b65c6-4930-4e2d-84f6-d139ece3032f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:56.306979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5c70c-2090-45a1-8b2f-9f568b22d5a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:56.309147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5c70c-2090-45a1-8b2f-9f568b22d5a7 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:56.319677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1064/2138] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:56.322334 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5c70c-2090-45a1-8b2f-9f568b22d5a7 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:56.324989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-24e5c70c-2090-45a1-8b2f-9f568b22d5a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:56.331865 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b65c6-4930-4e2d-84f6-d139ece3032f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:56.340379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5c70c-2090-45a1-8b2f-9f568b22d5a7 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:56.343043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e78ca0-3db8-41c3-a941-e67c3d2c66dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:56.344217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78ca0-3db8-41c3-a941-e67c3d2c66dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.344431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78ca0-3db8-41c3-a941-e67c3d2c66dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.345153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78ca0-3db8-41c3-a941-e67c3d2c66dc admin admin] PATH_INFO: `/v3/projects/7d1b32acf10e41978fcb731da8270425/users/6bc22e7f4fe74030abdb84f4eee10d5c/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.347818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e78ca0-3db8-41c3-a941-e67c3d2c66dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1b32acf10e41978fcb731da8270425, user_id=6bc22e7f4fe74030abdb84f4eee10d5c, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:56.351260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e78ca0-3db8-41c3-a941-e67c3d2c66dc admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:56.355839 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbf4e2-6216-46fc-9032-7e3e524e05c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:56.356592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbf4e2-6216-46fc-9032-7e3e524e05c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.356950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbf4e2-6216-46fc-9032-7e3e524e05c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.357304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbf4e2-6216-46fc-9032-7e3e524e05c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.360517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c3bbf4e2-6216-46fc-9032-7e3e524e05c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:56.378597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1075/2139] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:56 2022] PUT /identity/v3/projects/7d1b32acf10e41978fcb731da8270425/users/6bc22e7f4fe74030abdb84f4eee10d5c/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:56.389734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c5b29-63ce-48de-aa3b-0615106e8b0c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.390104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c5b29-63ce-48de-aa3b-0615106e8b0c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.390414 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c5b29-63ce-48de-aa3b-0615106e8b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.420791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-568c5b29-63ce-48de-aa3b-0615106e8b0c None None] MFA Rules not processed for user `6bc22e7f4fe74030abdb84f4eee10d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:56.422050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-568c5b29-63ce-48de-aa3b-0615106e8b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:56.447406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbf4e2-6216-46fc-9032-7e3e524e05c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:56.448896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbf4e2-6216-46fc-9032-7e3e524e05c9 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:56.458722 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1065/2140] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:56.459431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbf4e2-6216-46fc-9032-7e3e524e05c9 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:56.460807 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c3bbf4e2-6216-46fc-9032-7e3e524e05c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:56.479576 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbf4e2-6216-46fc-9032-7e3e524e05c9 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:56.485031 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:56.506767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1076/2141] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:15:56.562079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:56.563334 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.563620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.563896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] PATH_INFO: `/v3/projects/52c4fbe8c2dd4de3936143e9afe1727c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.565149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c4fbe8c2dd4de3936143e9afe1727c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:56.567225 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:56.619413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:56.628486 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c4fbe8c2dd4de3936143e9afe1727c', 'request_id': 'req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:56.644091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c4fbe8c2dd4de3936143e9afe1727c', 'request_id': 'req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:56.652984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c4fbe8c2dd4de3936143e9afe1727c', 'request_id': 'req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:56.665683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] The token cache is being invalidate because project 52c4fbe8c2dd4de3936143e9afe1727c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:15:56.666022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:56.666800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:15:56.668767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1066/2142] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:15:56 2022] DELETE /identity/v3/projects/52c4fbe8c2dd4de3936143e9afe1727c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:15:56.675948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc46d75-ce32-4dd2-a293-42bc3552eec7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.676430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc46d75-ce32-4dd2-a293-42bc3552eec7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.676715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc46d75-ce32-4dd2-a293-42bc3552eec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.697804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:56.697804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:56.703561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2dc46d75-ce32-4dd2-a293-42bc3552eec7 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:56.704806 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2dc46d75-ce32-4dd2-a293-42bc3552eec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:56.775628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aab36f-eb68-4480-bfbc-4406ccde2fe8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:56.778132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1077/2143] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:56.780854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a54868-0eea-417f-a870-a7a14facccc3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.781105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a54868-0eea-417f-a870-a7a14facccc3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.781382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a54868-0eea-417f-a870-a7a14facccc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.785741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cbaad-c04c-4b0a-8735-924b4aa6ca36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:56.786407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cbaad-c04c-4b0a-8735-924b4aa6ca36 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.786674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cbaad-c04c-4b0a-8735-924b4aa6ca36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.786944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cbaad-c04c-4b0a-8735-924b4aa6ca36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.791122 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e01cbaad-c04c-4b0a-8735-924b4aa6ca36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:56.809707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-99a54868-0eea-417f-a870-a7a14facccc3 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:56.811244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-99a54868-0eea-417f-a870-a7a14facccc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:56.824125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1078/2144] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:56.825371 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a54868-0eea-417f-a870-a7a14facccc3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:56.838142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a54868-0eea-417f-a870-a7a14facccc3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:56.844028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9605e-f773-41cf-99bb-0f4371c72e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:56.844817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9605e-f773-41cf-99bb-0f4371c72e9a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.845166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9605e-f773-41cf-99bb-0f4371c72e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.845579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9605e-f773-41cf-99bb-0f4371c72e9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.846579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9605e-f773-41cf-99bb-0f4371c72e9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:15:56.846956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9605e-f773-41cf-99bb-0f4371c72e9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:56.848982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9605e-f773-41cf-99bb-0f4371c72e9a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:56.866311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1079/2145] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:15:56.867869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9605e-f773-41cf-99bb-0f4371c72e9a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:56.879733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9605e-f773-41cf-99bb-0f4371c72e9a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:56.885320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe5d2e-df68-4fdc-a94d-5bd48a6054ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:56.886056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe5d2e-df68-4fdc-a94d-5bd48a6054ba admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:56.886339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe5d2e-df68-4fdc-a94d-5bd48a6054ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:56.886623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe5d2e-df68-4fdc-a94d-5bd48a6054ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:56.887336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe5d2e-df68-4fdc-a94d-5bd48a6054ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:56.889391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe5d2e-df68-4fdc-a94d-5bd48a6054ba admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:56.897108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cbaad-c04c-4b0a-8735-924b4aa6ca36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:56.898922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cbaad-c04c-4b0a-8735-924b4aa6ca36 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:56.911590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1067/2146] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:15:56.919944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1080/2147] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:15:56 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:56.924385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cbaad-c04c-4b0a-8735-924b4aa6ca36 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:56.950379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cbaad-c04c-4b0a-8735-924b4aa6ca36 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:57.016561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe5d2e-df68-4fdc-a94d-5bd48a6054ba admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:57.020932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07251f-6427-45ec-8898-871387226976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:57.021823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07251f-6427-45ec-8898-871387226976 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:57.022288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07251f-6427-45ec-8898-871387226976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:57.022704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07251f-6427-45ec-8898-871387226976 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:57.023552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07251f-6427-45ec-8898-871387226976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:57.025775 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07251f-6427-45ec-8898-871387226976 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:57.049428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1068/2148] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:15:56 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:15:57.053488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07251f-6427-45ec-8898-871387226976 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:57.067888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07251f-6427-45ec-8898-871387226976 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:57.073605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc01cd-fa13-4649-b7c9-7900b2f05660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:57.074372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc01cd-fa13-4649-b7c9-7900b2f05660 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:57.074656 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc01cd-fa13-4649-b7c9-7900b2f05660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:57.074958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc01cd-fa13-4649-b7c9-7900b2f05660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:57.075698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc01cd-fa13-4649-b7c9-7900b2f05660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:57.077324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc01cd-fa13-4649-b7c9-7900b2f05660 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:57.078915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:57.088936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe5d2e-df68-4fdc-a94d-5bd48a6054ba admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:57.094974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b47d4d-aaf8-49a1-8925-42e069cc2f91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:57.095854 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b47d4d-aaf8-49a1-8925-42e069cc2f91 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:57.096266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b47d4d-aaf8-49a1-8925-42e069cc2f91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:57.096579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b47d4d-aaf8-49a1-8925-42e069cc2f91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:57.098906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-65b47d4d-aaf8-49a1-8925-42e069cc2f91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:57.099688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1069/2149] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:15:57.103503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc01cd-fa13-4649-b7c9-7900b2f05660 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:57.114319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc01cd-fa13-4649-b7c9-7900b2f05660 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:57.119590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1666c1-d471-4d1c-b58d-d812e1aa8c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:57.120257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1666c1-d471-4d1c-b58d-d812e1aa8c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:57.120537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1666c1-d471-4d1c-b58d-d812e1aa8c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:57.120810 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1666c1-d471-4d1c-b58d-d812e1aa8c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:57.121527 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1666c1-d471-4d1c-b58d-d812e1aa8c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:57.123239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1666c1-d471-4d1c-b58d-d812e1aa8c4e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:57.135716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1070/2150] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:15:57.140131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1666c1-d471-4d1c-b58d-d812e1aa8c4e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:57.155558 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1666c1-d471-4d1c-b58d-d812e1aa8c4e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:57.155558 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbf1c4-1549-445b-aece-c2341d243b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:57.156227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbf1c4-1549-445b-aece-c2341d243b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:57.156514 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbf1c4-1549-445b-aece-c2341d243b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:57.156796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbf1c4-1549-445b-aece-c2341d243b56 admin admin] PATH_INFO: `/v3/projects/e0a2c7fa39d4475ea0e389ced1ecc89f/users/8a1617debf97457bb2b0aa85af67a8ac/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:57.172464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbf1c4-1549-445b-aece-c2341d243b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a2c7fa39d4475ea0e389ced1ecc89f, user_id=8a1617debf97457bb2b0aa85af67a8ac, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:57.175381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbf1c4-1549-445b-aece-c2341d243b56 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:57.177631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b47d4d-aaf8-49a1-8925-42e069cc2f91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:57.179319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b47d4d-aaf8-49a1-8925-42e069cc2f91 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:57.189665 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1081/2151] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:15:57.190640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1071/2152] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:15:57 2022] PUT /identity/v3/projects/e0a2c7fa39d4475ea0e389ced1ecc89f/users/8a1617debf97457bb2b0aa85af67a8ac/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:15:57.201766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395053f3-570f-47e1-bbc7-1c8a1ea4c136 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:57.202097 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395053f3-570f-47e1-bbc7-1c8a1ea4c136 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:57.202381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395053f3-570f-47e1-bbc7-1c8a1ea4c136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:57.210157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:57.231130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-395053f3-570f-47e1-bbc7-1c8a1ea4c136 None None] MFA Rules not processed for user `8a1617debf97457bb2b0aa85af67a8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:15:57.233079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-395053f3-570f-47e1-bbc7-1c8a1ea4c136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:57.299255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1082/2153] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:15:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:15:57.311742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbf1c4-1549-445b-aece-c2341d243b56 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:57.313548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9dcbf1c4-1549-445b-aece-c2341d243b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:57.374305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbf1c4-1549-445b-aece-c2341d243b56 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:57.378073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81837e-5508-4a7c-8d47-271efdee12b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:57.378733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81837e-5508-4a7c-8d47-271efdee12b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:57.379017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81837e-5508-4a7c-8d47-271efdee12b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:57.379290 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81837e-5508-4a7c-8d47-271efdee12b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:57.388664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81837e-5508-4a7c-8d47-271efdee12b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:57.390246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81837e-5508-4a7c-8d47-271efdee12b6 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:57.399485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1072/2154] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:58.095110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395053f3-570f-47e1-bbc7-1c8a1ea4c136 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:58.105972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395053f3-570f-47e1-bbc7-1c8a1ea4c136 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:58.109624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81261720-8b97-4436-97d1-931efbad9a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:58.110314 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81261720-8b97-4436-97d1-931efbad9a7a service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:58.110598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81261720-8b97-4436-97d1-931efbad9a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:58.110876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81261720-8b97-4436-97d1-931efbad9a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:58.113125 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-81261720-8b97-4436-97d1-931efbad9a7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:58.145734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81837e-5508-4a7c-8d47-271efdee12b6 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:58.155549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81837e-5508-4a7c-8d47-271efdee12b6 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:58.160252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd503704-ce20-43aa-8e60-3bdaeb588678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:58.161741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd503704-ce20-43aa-8e60-3bdaeb588678 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:58.161741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd503704-ce20-43aa-8e60-3bdaeb588678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:58.161958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd503704-ce20-43aa-8e60-3bdaeb588678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:58.164376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bd503704-ce20-43aa-8e60-3bdaeb588678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:58.192074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81261720-8b97-4436-97d1-931efbad9a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:58.193501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81261720-8b97-4436-97d1-931efbad9a7a service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:58.207464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1083/2155] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:58.263881 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd503704-ce20-43aa-8e60-3bdaeb588678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:58.266583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd503704-ce20-43aa-8e60-3bdaeb588678 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:58.279446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1073/2156] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:59.146319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81261720-8b97-4436-97d1-931efbad9a7a service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:59.155518 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81261720-8b97-4436-97d1-931efbad9a7a service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:59.159355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed708ed-7764-43b5-83b6-51c9768300f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:59.160013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed708ed-7764-43b5-83b6-51c9768300f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:59.160297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed708ed-7764-43b5-83b6-51c9768300f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:59.160572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed708ed-7764-43b5-83b6-51c9768300f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:59.162417 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0ed708ed-7764-43b5-83b6-51c9768300f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:59.237389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed708ed-7764-43b5-83b6-51c9768300f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:59.239194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed708ed-7764-43b5-83b6-51c9768300f2 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:59.252190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1084/2157] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:15:59.294674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd503704-ce20-43aa-8e60-3bdaeb588678 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:15:59.306313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd503704-ce20-43aa-8e60-3bdaeb588678 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:15:59.312704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991366b3-0c6a-4899-abca-e6d2ec3d2b06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:15:59.313638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991366b3-0c6a-4899-abca-e6d2ec3d2b06 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:15:59.314121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991366b3-0c6a-4899-abca-e6d2ec3d2b06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:15:59.314641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991366b3-0c6a-4899-abca-e6d2ec3d2b06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:15:59.316877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-991366b3-0c6a-4899-abca-e6d2ec3d2b06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:15:59.429936 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:15:59.435632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991366b3-0c6a-4899-abca-e6d2ec3d2b06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:15:59.437815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991366b3-0c6a-4899-abca-e6d2ec3d2b06 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:15:59.449611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1074/2158] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:15:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:00.966601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed708ed-7764-43b5-83b6-51c9768300f2 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:01.008659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:01.047737 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed708ed-7764-43b5-83b6-51c9768300f2 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:01.053385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:01.054320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:01.054601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:01.055041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] PATH_INFO: `/v3/users/8a1617debf97457bb2b0aa85af67a8ac` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:01.056287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a1617debf97457bb2b0aa85af67a8ac)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:01.058423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:01.114612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a1617debf97457bb2b0aa85af67a8ac'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:01.115545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a1617debf97457bb2b0aa85af67a8ac'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:01.124298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a1617debf97457bb2b0aa85af67a8ac'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:01.137382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a1617debf97457bb2b0aa85af67a8ac'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:01.138227 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a1617debf97457bb2b0aa85af67a8ac'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:01.166017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1085/2159] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:00 2022] DELETE /identity/v3/users/8a1617debf97457bb2b0aa85af67a8ac => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:01.171352 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4633f-3cdc-4d37-94b5-887dc3d8a008 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:01.171843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4633f-3cdc-4d37-94b5-887dc3d8a008 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:01.172093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4633f-3cdc-4d37-94b5-887dc3d8a008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:01.189922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-aae4633f-3cdc-4d37-94b5-887dc3d8a008 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:01.191242 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aae4633f-3cdc-4d37-94b5-887dc3d8a008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:01.270466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1075/2160] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:01.283606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:01.285140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:01.349693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf9e8b-6606-455f-bee9-ba5472807c2c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:01.358074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd6b39-f394-478c-95c4-274f501d5613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:01.358924 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd6b39-f394-478c-95c4-274f501d5613 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:01.359359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd6b39-f394-478c-95c4-274f501d5613 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:01.359642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd6b39-f394-478c-95c4-274f501d5613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:01.369979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd6b39-f394-478c-95c4-274f501d5613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:01.371437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd6b39-f394-478c-95c4-274f501d5613 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:01.381534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1086/2161] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:01.680416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4633f-3cdc-4d37-94b5-887dc3d8a008 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:01.682033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aae4633f-3cdc-4d37-94b5-887dc3d8a008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:01.694704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4633f-3cdc-4d37-94b5-887dc3d8a008 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:01.699565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:01.700448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:01.700732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:01.701009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] PATH_INFO: `/v3/projects/e0a2c7fa39d4475ea0e389ced1ecc89f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:01.702280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a2c7fa39d4475ea0e389ced1ecc89f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:01.704436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:01.754080 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a2c7fa39d4475ea0e389ced1ecc89f', 'request_id': 'req-4f2ed5af-d694-4822-8bd5-aba04df98fb8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:01.763204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a2c7fa39d4475ea0e389ced1ecc89f', 'request_id': 'req-4f2ed5af-d694-4822-8bd5-aba04df98fb8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:01.772301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a2c7fa39d4475ea0e389ced1ecc89f', 'request_id': 'req-4f2ed5af-d694-4822-8bd5-aba04df98fb8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:01.781791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] The token cache is being invalidate because project e0a2c7fa39d4475ea0e389ced1ecc89f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:01.782119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:01.782817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4f2ed5af-d694-4822-8bd5-aba04df98fb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:01.784596 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1076/2162] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:01 2022] DELETE /identity/v3/projects/e0a2c7fa39d4475ea0e389ced1ecc89f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:01.791385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113b567-5eb5-409c-9a4d-be670a147e71 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:01.791676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113b567-5eb5-409c-9a4d-be670a147e71 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:01.791955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113b567-5eb5-409c-9a4d-be670a147e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:01.811444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d113b567-5eb5-409c-9a4d-be670a147e71 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:01.812237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d113b567-5eb5-409c-9a4d-be670a147e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:01.868461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:01.891255 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1087/2163] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:01.900375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6865c8-aaf3-4fd1-9176-8c1ed5306fd1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:01.900478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6865c8-aaf3-4fd1-9176-8c1ed5306fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:01.900746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6865c8-aaf3-4fd1-9176-8c1ed5306fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:01.935598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3e6865c8-aaf3-4fd1-9176-8c1ed5306fd1 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:01.937051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3e6865c8-aaf3-4fd1-9176-8c1ed5306fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:01.952334 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1077/2164] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:01.953690 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113b567-5eb5-409c-9a4d-be670a147e71 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:01.971090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113b567-5eb5-409c-9a4d-be670a147e71 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:01.979261 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eced5-173d-4181-8d70-c6486bd5475b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:01.980294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eced5-173d-4181-8d70-c6486bd5475b admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:01.981222 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eced5-173d-4181-8d70-c6486bd5475b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:01.981869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eced5-173d-4181-8d70-c6486bd5475b admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:01.983090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eced5-173d-4181-8d70-c6486bd5475b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:01.983679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eced5-173d-4181-8d70-c6486bd5475b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:01.986134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eced5-173d-4181-8d70-c6486bd5475b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:02.003308 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1088/2165] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:16:02.007532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6865c8-aaf3-4fd1-9176-8c1ed5306fd1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:02.017299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6865c8-aaf3-4fd1-9176-8c1ed5306fd1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:02.022651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d6dfe-621a-41d5-8f58-90b3188f2701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:02.023262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d6dfe-621a-41d5-8f58-90b3188f2701 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:02.023547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d6dfe-621a-41d5-8f58-90b3188f2701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:02.023828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d6dfe-621a-41d5-8f58-90b3188f2701 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:02.024541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d6dfe-621a-41d5-8f58-90b3188f2701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:02.026611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d6dfe-621a-41d5-8f58-90b3188f2701 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:02.062114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1078/2166] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:02 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:02.063298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eced5-173d-4181-8d70-c6486bd5475b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:02.080142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eced5-173d-4181-8d70-c6486bd5475b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:02.168940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0ac0b-612c-4308-b369-00fb73eb9fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:02.169953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0ac0b-612c-4308-b369-00fb73eb9fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:02.170393 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0ac0b-612c-4308-b369-00fb73eb9fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:02.170701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0ac0b-612c-4308-b369-00fb73eb9fb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:02.171381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0ac0b-612c-4308-b369-00fb73eb9fb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:02.173472 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0ac0b-612c-4308-b369-00fb73eb9fb8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:02.213486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1089/2167] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:02 2022] POST /identity/v3/users => generated 286 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:02.214051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d6dfe-621a-41d5-8f58-90b3188f2701 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:02.225344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d6dfe-621a-41d5-8f58-90b3188f2701 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:02.230047 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f624e-93d2-4af0-baa1-77fbe9d14b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:02.230713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f624e-93d2-4af0-baa1-77fbe9d14b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:02.231021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f624e-93d2-4af0-baa1-77fbe9d14b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:02.231306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f624e-93d2-4af0-baa1-77fbe9d14b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:02.232188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f624e-93d2-4af0-baa1-77fbe9d14b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:02.233820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f624e-93d2-4af0-baa1-77fbe9d14b95 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:02.246147 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1079/2168] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:02.250214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0ac0b-612c-4308-b369-00fb73eb9fb8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:02.261597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0ac0b-612c-4308-b369-00fb73eb9fb8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:02.266402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f1c2e-6004-496b-ad68-962b51dd77b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:02.267065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f1c2e-6004-496b-ad68-962b51dd77b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:02.267353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f1c2e-6004-496b-ad68-962b51dd77b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:02.267886 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f1c2e-6004-496b-ad68-962b51dd77b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:02.268621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f1c2e-6004-496b-ad68-962b51dd77b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:02.270410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f1c2e-6004-496b-ad68-962b51dd77b3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:02.288133 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1090/2169] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:16:02.292250 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f624e-93d2-4af0-baa1-77fbe9d14b95 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:02.306304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f624e-93d2-4af0-baa1-77fbe9d14b95 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:02.311979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cccda1-08a6-4700-9f24-f942e133947c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:02.314150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cccda1-08a6-4700-9f24-f942e133947c admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:02.314464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cccda1-08a6-4700-9f24-f942e133947c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:02.314750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cccda1-08a6-4700-9f24-f942e133947c admin admin] PATH_INFO: `/v3/projects/33b3e0ba3aa644ddbbee8f56be21ad06/users/5ba79c77fcd54bbcb673a5cba1ed2975/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:02.333951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cccda1-08a6-4700-9f24-f942e133947c admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b3e0ba3aa644ddbbee8f56be21ad06, user_id=5ba79c77fcd54bbcb673a5cba1ed2975, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:02.337008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cccda1-08a6-4700-9f24-f942e133947c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:02.356531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1080/2170] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:02 2022] PUT /identity/v3/projects/33b3e0ba3aa644ddbbee8f56be21ad06/users/5ba79c77fcd54bbcb673a5cba1ed2975/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:02.361029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788da23e-442a-4495-abaa-b93fe0c5eddf None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:02.361224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788da23e-442a-4495-abaa-b93fe0c5eddf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:02.361573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788da23e-442a-4495-abaa-b93fe0c5eddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:02.381664 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-788da23e-442a-4495-abaa-b93fe0c5eddf None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:02.383088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-788da23e-442a-4495-abaa-b93fe0c5eddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:02.456134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cccda1-08a6-4700-9f24-f942e133947c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:02.458781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c9cccda1-08a6-4700-9f24-f942e133947c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:02.467598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1091/2171] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:02.472573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cccda1-08a6-4700-9f24-f942e133947c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:02.475335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788da23e-442a-4495-abaa-b93fe0c5eddf None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:02.476757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-788da23e-442a-4495-abaa-b93fe0c5eddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:02.477961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e90c3-819d-4a71-8f2b-b76f3febc085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:02.480104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e90c3-819d-4a71-8f2b-b76f3febc085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:02.480336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e90c3-819d-4a71-8f2b-b76f3febc085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:02.483865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e90c3-819d-4a71-8f2b-b76f3febc085 admin admin] PATH_INFO: `/v3/users/3fa9bb26cf6546419c4e2a64ca44fba6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:02.483865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e90c3-819d-4a71-8f2b-b76f3febc085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa9bb26cf6546419c4e2a64ca44fba6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:02.485800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e90c3-819d-4a71-8f2b-b76f3febc085 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:02.498550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:02.546175 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b9e90c3-819d-4a71-8f2b-b76f3febc085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa9bb26cf6546419c4e2a64ca44fba6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:02.546487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788da23e-442a-4495-abaa-b93fe0c5eddf None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:02.546772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b9e90c3-819d-4a71-8f2b-b76f3febc085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa9bb26cf6546419c4e2a64ca44fba6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:02.547384 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b9e90c3-819d-4a71-8f2b-b76f3febc085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa9bb26cf6546419c4e2a64ca44fba6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:02.550553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b413fa-a3eb-42b7-b25d-23dc9c48022b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:02.551242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b413fa-a3eb-42b7-b25d-23dc9c48022b service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:02.551671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b413fa-a3eb-42b7-b25d-23dc9c48022b service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:02.551950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b413fa-a3eb-42b7-b25d-23dc9c48022b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:02.554003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-59b413fa-a3eb-42b7-b25d-23dc9c48022b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:02.555398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b9e90c3-819d-4a71-8f2b-b76f3febc085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa9bb26cf6546419c4e2a64ca44fba6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:02.562624 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9b9e90c3-819d-4a71-8f2b-b76f3febc085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa9bb26cf6546419c4e2a64ca44fba6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:02.566074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b413fa-a3eb-42b7-b25d-23dc9c48022b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:02.567545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b413fa-a3eb-42b7-b25d-23dc9c48022b service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:02.571535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:02.580009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1081/2172] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:02 2022] DELETE /identity/v3/users/3fa9bb26cf6546419c4e2a64ca44fba6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:02.584502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e8971-56cd-4157-b221-e6202b92840e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:02.584796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e8971-56cd-4157-b221-e6202b92840e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:02.585076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e8971-56cd-4157-b221-e6202b92840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:02.586073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1092/2173] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:16:02.602326 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b02e8971-56cd-4157-b221-e6202b92840e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:02.603616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b02e8971-56cd-4157-b221-e6202b92840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:02.674839 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1082/2174] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:16:02.686355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b413fa-a3eb-42b7-b25d-23dc9c48022b service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:02.689120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-59b413fa-a3eb-42b7-b25d-23dc9c48022b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:02.701951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b413fa-a3eb-42b7-b25d-23dc9c48022b service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:02.782070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0866ca2-2920-4bb3-b910-d0f2a25c2303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:02.782861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0866ca2-2920-4bb3-b910-d0f2a25c2303 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:02.783154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0866ca2-2920-4bb3-b910-d0f2a25c2303 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:02.783476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0866ca2-2920-4bb3-b910-d0f2a25c2303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:02.797943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0866ca2-2920-4bb3-b910-d0f2a25c2303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:02.799864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0866ca2-2920-4bb3-b910-d0f2a25c2303 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:02.813934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1093/2175] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:03.122670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e8971-56cd-4157-b221-e6202b92840e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:03.124190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b02e8971-56cd-4157-b221-e6202b92840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:03.144773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e8971-56cd-4157-b221-e6202b92840e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:03.149739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:03.157433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:03.157727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:03.158038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] PATH_INFO: `/v3/projects/4ab8e41403bb426bb54d265fa98bdbb2` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:03.159912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ab8e41403bb426bb54d265fa98bdbb2)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:03.161636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:03.222993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ab8e41403bb426bb54d265fa98bdbb2', 'request_id': 'req-49394db9-14f7-4db1-b6b6-9d782b7911e5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.234329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ab8e41403bb426bb54d265fa98bdbb2', 'request_id': 'req-49394db9-14f7-4db1-b6b6-9d782b7911e5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.259647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ab8e41403bb426bb54d265fa98bdbb2', 'request_id': 'req-49394db9-14f7-4db1-b6b6-9d782b7911e5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.297149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0866ca2-2920-4bb3-b910-d0f2a25c2303 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:03.297647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] The token cache is being invalidate because project 4ab8e41403bb426bb54d265fa98bdbb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:03.297822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.298927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f0866ca2-2920-4bb3-b910-d0f2a25c2303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:03.300205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.303149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1083/2176] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:03 2022] DELETE /identity/v3/projects/4ab8e41403bb426bb54d265fa98bdbb2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:03.306665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:03.308675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:03.312850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0866ca2-2920-4bb3-b910-d0f2a25c2303 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:03.328358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:03.406089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49394db9-14f7-4db1-b6b6-9d782b7911e5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:03.412445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490379e-7610-4258-be6c-7debfb535f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:03.413458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490379e-7610-4258-be6c-7debfb535f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:03.413780 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490379e-7610-4258-be6c-7debfb535f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:03.414192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490379e-7610-4258-be6c-7debfb535f1a admin admin] PATH_INFO: `/v3/users/6bc22e7f4fe74030abdb84f4eee10d5c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:03.415784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490379e-7610-4258-be6c-7debfb535f1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc22e7f4fe74030abdb84f4eee10d5c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:03.417782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490379e-7610-4258-be6c-7debfb535f1a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:03.450421 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:03.451325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:03.451609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:03.452051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] PATH_INFO: `/v3/users/721fcdf3811d4d37ab1304ba1972ff5f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:03.457554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=721fcdf3811d4d37ab1304ba1972ff5f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:03.459587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:03.523737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2490379e-7610-4258-be6c-7debfb535f1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc22e7f4fe74030abdb84f4eee10d5c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.524713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2490379e-7610-4258-be6c-7debfb535f1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc22e7f4fe74030abdb84f4eee10d5c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.525425 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2490379e-7610-4258-be6c-7debfb535f1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc22e7f4fe74030abdb84f4eee10d5c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.549943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2490379e-7610-4258-be6c-7debfb535f1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc22e7f4fe74030abdb84f4eee10d5c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.558161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '721fcdf3811d4d37ab1304ba1972ff5f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.558925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '721fcdf3811d4d37ab1304ba1972ff5f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.563013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2490379e-7610-4258-be6c-7debfb535f1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc22e7f4fe74030abdb84f4eee10d5c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.584600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '721fcdf3811d4d37ab1304ba1972ff5f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.593934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '721fcdf3811d4d37ab1304ba1972ff5f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.594698 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '721fcdf3811d4d37ab1304ba1972ff5f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.615853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1084/2177] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:03 2022] DELETE /identity/v3/users/6bc22e7f4fe74030abdb84f4eee10d5c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:03.628721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1094/2178] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:03 2022] DELETE /identity/v3/users/721fcdf3811d4d37ab1304ba1972ff5f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:16:03.633837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29261bb5-4e0f-47e4-bd38-34a9b0c791c3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:03.634446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29261bb5-4e0f-47e4-bd38-34a9b0c791c3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:03.634733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29261bb5-4e0f-47e4-bd38-34a9b0c791c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:03.637256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:03.643184 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:03.669653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-29261bb5-4e0f-47e4-bd38-34a9b0c791c3 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:03.672460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-29261bb5-4e0f-47e4-bd38-34a9b0c791c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:03.687913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:03.788945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de1c29-da31-49ce-a037-579a57b77e7b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:03.814928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518b80e-cb20-4621-9a93-3f025665cbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:03.815822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1085/2179] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:03.821324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518b80e-cb20-4621-9a93-3f025665cbe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:03.822154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518b80e-cb20-4621-9a93-3f025665cbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:03.822610 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518b80e-cb20-4621-9a93-3f025665cbe3 admin admin] PATH_INFO: `/v3/users/409a822e17d84649be4ba569b2cb3e2e` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:03.825457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518b80e-cb20-4621-9a93-3f025665cbe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=409a822e17d84649be4ba569b2cb3e2e)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:03.828626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518b80e-cb20-4621-9a93-3f025665cbe3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:03.834377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29261bb5-4e0f-47e4-bd38-34a9b0c791c3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:03.835830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-29261bb5-4e0f-47e4-bd38-34a9b0c791c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:03.894777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:03.928774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7518b80e-cb20-4621-9a93-3f025665cbe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409a822e17d84649be4ba569b2cb3e2e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.929606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7518b80e-cb20-4621-9a93-3f025665cbe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409a822e17d84649be4ba569b2cb3e2e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.941016 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7518b80e-cb20-4621-9a93-3f025665cbe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409a822e17d84649be4ba569b2cb3e2e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.954569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7518b80e-cb20-4621-9a93-3f025665cbe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409a822e17d84649be4ba569b2cb3e2e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.956561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7518b80e-cb20-4621-9a93-3f025665cbe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409a822e17d84649be4ba569b2cb3e2e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:03.960882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29261bb5-4e0f-47e4-bd38-34a9b0c791c3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:03.971389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557897c-ed5d-48b9-9682-6be724ed214d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:03.974322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557897c-ed5d-48b9-9682-6be724ed214d service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:03.974648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557897c-ed5d-48b9-9682-6be724ed214d service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:03.974932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557897c-ed5d-48b9-9682-6be724ed214d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:03.979966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e557897c-ed5d-48b9-9682-6be724ed214d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:03.988502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1095/2180] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:03 2022] DELETE /identity/v3/users/409a822e17d84649be4ba569b2cb3e2e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:03.994586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b400e41-cd80-4c22-94c9-3f387a15e58c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:03.994951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b400e41-cd80-4c22-94c9-3f387a15e58c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:03.995716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b400e41-cd80-4c22-94c9-3f387a15e58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:04.017817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e557897c-ed5d-48b9-9682-6be724ed214d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:04.020122 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e557897c-ed5d-48b9-9682-6be724ed214d service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:04.028688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-6b400e41-cd80-4c22-94c9-3f387a15e58c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:04.031412 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6b400e41-cd80-4c22-94c9-3f387a15e58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:04.045580 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1086/2181] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:04.118451 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557897c-ed5d-48b9-9682-6be724ed214d service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:04.120750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e557897c-ed5d-48b9-9682-6be724ed214d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:04.144083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1096/2182] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:04.146299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557897c-ed5d-48b9-9682-6be724ed214d service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:04.159725 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b400e41-cd80-4c22-94c9-3f387a15e58c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:04.161155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6b400e41-cd80-4c22-94c9-3f387a15e58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:04.178369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:04.179255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:04.179541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:04.179815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] PATH_INFO: `/v3/projects/7d1b32acf10e41978fcb731da8270425` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:04.183579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d1b32acf10e41978fcb731da8270425)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:04.189511 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:04.268797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:04.285385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1b32acf10e41978fcb731da8270425', 'request_id': 'req-46ee9e41-d8f8-4d40-81ce-66ed39eed397'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:04.296299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1b32acf10e41978fcb731da8270425', 'request_id': 'req-46ee9e41-d8f8-4d40-81ce-66ed39eed397'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:04.305539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1b32acf10e41978fcb731da8270425', 'request_id': 'req-46ee9e41-d8f8-4d40-81ce-66ed39eed397'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:04.324113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b400e41-cd80-4c22-94c9-3f387a15e58c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:04.324587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] The token cache is being invalidate because project 7d1b32acf10e41978fcb731da8270425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:04.325922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:04.326828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-46ee9e41-d8f8-4d40-81ce-66ed39eed397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:04.335025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1087/2183] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:04 2022] DELETE /identity/v3/projects/7d1b32acf10e41978fcb731da8270425 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:16:04.356497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8760a-7bb4-4d96-b4b5-eabcd2143a6a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:04.360765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8760a-7bb4-4d96-b4b5-eabcd2143a6a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:04.361182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8760a-7bb4-4d96-b4b5-eabcd2143a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:04.394070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-90a8760a-7bb4-4d96-b4b5-eabcd2143a6a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:04.398063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-90a8760a-7bb4-4d96-b4b5-eabcd2143a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:04.399822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f7197-220f-4739-9009-e037c6c5cc27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:04.400670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f7197-220f-4739-9009-e037c6c5cc27 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:04.400967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f7197-220f-4739-9009-e037c6c5cc27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:04.401242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f7197-220f-4739-9009-e037c6c5cc27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:04.405254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ce5f7197-220f-4739-9009-e037c6c5cc27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:04.503423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1088/2184] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:04.510343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a8760a-7bb4-4d96-b4b5-eabcd2143a6a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:04.512474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-90a8760a-7bb4-4d96-b4b5-eabcd2143a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:04.536242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f7197-220f-4739-9009-e037c6c5cc27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:04.537800 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f7197-220f-4739-9009-e037c6c5cc27 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:04.549743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a8760a-7bb4-4d96-b4b5-eabcd2143a6a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:04.550062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1097/2185] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:16:04.554796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195b6d6-50a9-4ce4-86e7-3d91fe00a214 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:04.555089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195b6d6-50a9-4ce4-86e7-3d91fe00a214 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:04.555373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195b6d6-50a9-4ce4-86e7-3d91fe00a214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:04.576247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:04.577348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:04.578253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:04.578716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] PATH_INFO: `/v3/projects/88b5d59895d441ffb0122673f9131bad` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:04.584379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b5d59895d441ffb0122673f9131bad)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:04.586859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:04.589562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5195b6d6-50a9-4ce4-86e7-3d91fe00a214 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:04.592058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5195b6d6-50a9-4ce4-86e7-3d91fe00a214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:04.607267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1098/2186] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:04.619879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195b6d6-50a9-4ce4-86e7-3d91fe00a214 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:04.653909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195b6d6-50a9-4ce4-86e7-3d91fe00a214 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:04.667769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a23af2-df98-4dcc-abc0-e8f88d7030d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:04.668730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a23af2-df98-4dcc-abc0-e8f88d7030d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:04.669022 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a23af2-df98-4dcc-abc0-e8f88d7030d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:04.669297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a23af2-df98-4dcc-abc0-e8f88d7030d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:04.672856 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a23af2-df98-4dcc-abc0-e8f88d7030d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:04.674194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a23af2-df98-4dcc-abc0-e8f88d7030d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:04.680293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a23af2-df98-4dcc-abc0-e8f88d7030d1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:04.724575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1099/2187] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:16:04.730383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a23af2-df98-4dcc-abc0-e8f88d7030d1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:04.739690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b5d59895d441ffb0122673f9131bad', 'request_id': 'req-1da03135-f847-4950-8f31-685f48b3248f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:04.746860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:04.758378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b5d59895d441ffb0122673f9131bad', 'request_id': 'req-1da03135-f847-4950-8f31-685f48b3248f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:04.771036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b5d59895d441ffb0122673f9131bad', 'request_id': 'req-1da03135-f847-4950-8f31-685f48b3248f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:04.771793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a23af2-df98-4dcc-abc0-e8f88d7030d1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:04.794714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] The token cache is being invalidate because project 88b5d59895d441ffb0122673f9131bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:04.795085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:04.795854 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:04.802071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1089/2188] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:04 2022] DELETE /identity/v3/projects/88b5d59895d441ffb0122673f9131bad => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:04.803319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:04.806020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:04.948880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da03135-f847-4950-8f31-685f48b3248f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:04.961146 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:04.962981 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:04.963265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:04.963539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] PATH_INFO: `/v3/users/ca0c6d6bd5c243dc8f181de43ef1e05e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:04.965872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0c6d6bd5c243dc8f181de43ef1e05e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:04.968798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:04.974510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac5c1b-4cef-4bd1-b024-6c499da0d290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:04.975915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac5c1b-4cef-4bd1-b024-6c499da0d290 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:04.977552 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac5c1b-4cef-4bd1-b024-6c499da0d290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:04.979556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac5c1b-4cef-4bd1-b024-6c499da0d290 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:04.980444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac5c1b-4cef-4bd1-b024-6c499da0d290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:04.986605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac5c1b-4cef-4bd1-b024-6c499da0d290 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:05.032349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1100/2189] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:04 2022] POST /identity/v3/projects => generated 371 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:05.036526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac5c1b-4cef-4bd1-b024-6c499da0d290 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:05.039108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8eac5c1b-4cef-4bd1-b024-6c499da0d290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:05.048655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0c6d6bd5c243dc8f181de43ef1e05e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.051692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0c6d6bd5c243dc8f181de43ef1e05e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.052867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0c6d6bd5c243dc8f181de43ef1e05e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.054157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:05.070864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0c6d6bd5c243dc8f181de43ef1e05e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.084098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0c6d6bd5c243dc8f181de43ef1e05e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.119917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1090/2190] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:04 2022] DELETE /identity/v3/users/ca0c6d6bd5c243dc8f181de43ef1e05e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:05.175111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac5c1b-4cef-4bd1-b024-6c499da0d290 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:05.249585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08150a8-e8d1-436e-901b-9bfa7455c7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:05.250343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08150a8-e8d1-436e-901b-9bfa7455c7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:05.250674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08150a8-e8d1-436e-901b-9bfa7455c7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:05.250951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08150a8-e8d1-436e-901b-9bfa7455c7d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:05.251843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08150a8-e8d1-436e-901b-9bfa7455c7d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:05.255042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08150a8-e8d1-436e-901b-9bfa7455c7d1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:05.287846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1101/2191] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:05 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:16:05.292817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:05.294565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:05.308750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd3692-bb0f-4c56-997d-438242c169fc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:05.324650 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e651a-01fb-4a27-83c3-4d5d7076a9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:05.325451 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e651a-01fb-4a27-83c3-4d5d7076a9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:05.325732 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e651a-01fb-4a27-83c3-4d5d7076a9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:05.326091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e651a-01fb-4a27-83c3-4d5d7076a9f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:05.326827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e651a-01fb-4a27-83c3-4d5d7076a9f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:05.328880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e651a-01fb-4a27-83c3-4d5d7076a9f1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:05.340087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08150a8-e8d1-436e-901b-9bfa7455c7d1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:05.341963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c08150a8-e8d1-436e-901b-9bfa7455c7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:05.344111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1091/2192] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:05.345020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e651a-01fb-4a27-83c3-4d5d7076a9f1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:05.359032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08150a8-e8d1-436e-901b-9bfa7455c7d1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:05.366353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e651a-01fb-4a27-83c3-4d5d7076a9f1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:05.373544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed92850-0b63-43da-a4e7-61dc49d4f227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:05.374594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed92850-0b63-43da-a4e7-61dc49d4f227 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:05.375481 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed92850-0b63-43da-a4e7-61dc49d4f227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:05.375964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed92850-0b63-43da-a4e7-61dc49d4f227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:05.377116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed92850-0b63-43da-a4e7-61dc49d4f227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:05.379282 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed92850-0b63-43da-a4e7-61dc49d4f227 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:05.385355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:05.386275 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:05.387018 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:05.387018 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] PATH_INFO: `/v3/projects/bab6831ffd9e4097b440e55aadf9bef8` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:05.393157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bab6831ffd9e4097b440e55aadf9bef8)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:05.399563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1092/2193] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:05.399748 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:05.401653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed92850-0b63-43da-a4e7-61dc49d4f227 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:05.414798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed92850-0b63-43da-a4e7-61dc49d4f227 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:05.423935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5f592-2b46-4360-b85c-3279ddacf6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:05.424484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5f592-2b46-4360-b85c-3279ddacf6e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:05.424773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5f592-2b46-4360-b85c-3279ddacf6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:05.425124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5f592-2b46-4360-b85c-3279ddacf6e8 admin admin] PATH_INFO: `/v3/projects/77f9faeaa8334099ba0c171b8250b78c/users/85894c323fa94bcbb5585a7ae85cc355/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:05.448891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5f592-2b46-4360-b85c-3279ddacf6e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f9faeaa8334099ba0c171b8250b78c, user_id=85894c323fa94bcbb5585a7ae85cc355, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:05.451592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5f592-2b46-4360-b85c-3279ddacf6e8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:05.468600 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1093/2194] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:05 2022] PUT /identity/v3/projects/77f9faeaa8334099ba0c171b8250b78c/users/85894c323fa94bcbb5585a7ae85cc355/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:05.474414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c3dd7-6448-43d2-ab70-730a966570dc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:05.474821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c3dd7-6448-43d2-ab70-730a966570dc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:05.475325 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c3dd7-6448-43d2-ab70-730a966570dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:05.497429 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-fb9c3dd7-6448-43d2-ab70-730a966570dc None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:05.499095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fb9c3dd7-6448-43d2-ab70-730a966570dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:05.510599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:05.519937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bab6831ffd9e4097b440e55aadf9bef8', 'request_id': 'req-74602d14-0d73-4632-9a7b-81692d983edb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.535963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bab6831ffd9e4097b440e55aadf9bef8', 'request_id': 'req-74602d14-0d73-4632-9a7b-81692d983edb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.547053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bab6831ffd9e4097b440e55aadf9bef8', 'request_id': 'req-74602d14-0d73-4632-9a7b-81692d983edb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.572295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] The token cache is being invalidate because project bab6831ffd9e4097b440e55aadf9bef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:05.572908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.574871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.577618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1102/2195] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:05 2022] DELETE /identity/v3/projects/bab6831ffd9e4097b440e55aadf9bef8 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:05.584924 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:05.590875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:05.601005 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1094/2196] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:05.617660 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74602d14-0d73-4632-9a7b-81692d983edb admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:05.630480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:05.630964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c3dd7-6448-43d2-ab70-730a966570dc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:05.632207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:05.632424 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fb9c3dd7-6448-43d2-ab70-730a966570dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:05.632876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:05.633736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] PATH_INFO: `/v3/users/42de3812651a4bb893ff3ffae706718e` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:05.635763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=42de3812651a4bb893ff3ffae706718e)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:05.638373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:05.689582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:05.742213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c3dd7-6448-43d2-ab70-730a966570dc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:05.743250 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42de3812651a4bb893ff3ffae706718e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.743972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42de3812651a4bb893ff3ffae706718e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.757014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76190156-88d0-4500-9075-5e9dd87ed45b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:05.757760 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76190156-88d0-4500-9075-5e9dd87ed45b service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:05.758210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76190156-88d0-4500-9075-5e9dd87ed45b service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:05.758589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76190156-88d0-4500-9075-5e9dd87ed45b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:05.762311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42de3812651a4bb893ff3ffae706718e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.763220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-76190156-88d0-4500-9075-5e9dd87ed45b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:05.771064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42de3812651a4bb893ff3ffae706718e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.771684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42de3812651a4bb893ff3ffae706718e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:05.788412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76190156-88d0-4500-9075-5e9dd87ed45b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:05.791691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76190156-88d0-4500-9075-5e9dd87ed45b service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:05.796639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1103/2197] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:05 2022] DELETE /identity/v3/users/42de3812651a4bb893ff3ffae706718e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:05.796639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:05.808394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:05.819394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1095/2198] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:05.966215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1e3e0-6927-48a6-beda-eeae87936cca admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:05.978205 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:05.979034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:05.979315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:05.979610 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] PATH_INFO: `/v3/projects/f184d0a0a46b405898c17059cdb26a9b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:05.985084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f184d0a0a46b405898c17059cdb26a9b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:05.987680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:06.137383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f184d0a0a46b405898c17059cdb26a9b', 'request_id': 'req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:06.168399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f184d0a0a46b405898c17059cdb26a9b', 'request_id': 'req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:06.212598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f184d0a0a46b405898c17059cdb26a9b', 'request_id': 'req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:06.256609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] The token cache is being invalidate because project f184d0a0a46b405898c17059cdb26a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:06.258505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:06.263192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:06.272720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76190156-88d0-4500-9075-5e9dd87ed45b service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:06.279059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1104/2199] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:05 2022] DELETE /identity/v3/projects/f184d0a0a46b405898c17059cdb26a9b => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:06.288395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76190156-88d0-4500-9075-5e9dd87ed45b service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:06.451012 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:06.452148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:06.452438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:06.452808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] PATH_INFO: `/v3/users/f694f84f0e464d3bb8fd3ce3ca4d2c8c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:06.456771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f694f84f0e464d3bb8fd3ce3ca4d2c8c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:06.458944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:06.573910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f694f84f0e464d3bb8fd3ce3ca4d2c8c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:06.575749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f694f84f0e464d3bb8fd3ce3ca4d2c8c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:06.577761 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f694f84f0e464d3bb8fd3ce3ca4d2c8c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:06.595211 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f694f84f0e464d3bb8fd3ce3ca4d2c8c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:06.612289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:06.623294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f694f84f0e464d3bb8fd3ce3ca4d2c8c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:06.658939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1096/2200] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:06 2022] DELETE /identity/v3/users/f694f84f0e464d3bb8fd3ce3ca4d2c8c => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:06.796568 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:06.805030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:06.856716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ed4d4-12e2-4c13-98c2-6a34bf0dc7e4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:06.900486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:06.945141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:06.947684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:06.948206 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:06.948598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] PATH_INFO: `/v3/projects/c2cc190d6c8b45ab84e797235dead675` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:06.951380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2cc190d6c8b45ab84e797235dead675)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:06.954074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:07.032242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2cc190d6c8b45ab84e797235dead675', 'request_id': 'req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.046207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2cc190d6c8b45ab84e797235dead675', 'request_id': 'req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.057133 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2cc190d6c8b45ab84e797235dead675', 'request_id': 'req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.080605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] The token cache is being invalidate because project c2cc190d6c8b45ab84e797235dead675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:07.110519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.115330 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7fe41dfa-a6a2-4fa6-bee4-ec297befda40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.122529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1105/2201] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:06 2022] DELETE /identity/v3/projects/c2cc190d6c8b45ab84e797235dead675 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:07.124125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:07.126365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:07.151213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791ea2-8644-45fd-87bf-c01acc244c97 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:07.259631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:07.261223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:07.261519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:07.261805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] PATH_INFO: `/v3/users/8dad2caf1fac48b5b67e248030036e52` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:07.263173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dad2caf1fac48b5b67e248030036e52)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:07.265560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:07.354237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dad2caf1fac48b5b67e248030036e52'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.355533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dad2caf1fac48b5b67e248030036e52'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.356325 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dad2caf1fac48b5b67e248030036e52'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.370679 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dad2caf1fac48b5b67e248030036e52'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.405217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dad2caf1fac48b5b67e248030036e52'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.431812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1097/2202] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:07 2022] DELETE /identity/v3/users/8dad2caf1fac48b5b67e248030036e52 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:07.612650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353b091-b96d-409e-a1e9-95eae491718d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:07.614777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353b091-b96d-409e-a1e9-95eae491718d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:07.615629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353b091-b96d-409e-a1e9-95eae491718d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:07.661109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2353b091-b96d-409e-a1e9-95eae491718d None None] MFA Rules not processed for user `85894c323fa94bcbb5585a7ae85cc355`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:07.663485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2353b091-b96d-409e-a1e9-95eae491718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:07.699627 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:07.701129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:07.807703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:07.813484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aba972-5673-42aa-a982-f30e069b7415 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:07.817379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1106/2203] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:07.822081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353b091-b96d-409e-a1e9-95eae491718d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:07.824557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2353b091-b96d-409e-a1e9-95eae491718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:07.824874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:07.826157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:07.826485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:07.827185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] PATH_INFO: `/v3/projects/e8358c6201134e52a0ab36146cb470b7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:07.828969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8358c6201134e52a0ab36146cb470b7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:07.831632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:07.845253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353b091-b96d-409e-a1e9-95eae491718d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:07.854401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05babf41-64a1-4c00-a8c2-843060617593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:07.855279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05babf41-64a1-4c00-a8c2-843060617593 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:07.855890 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05babf41-64a1-4c00-a8c2-843060617593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:07.856658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05babf41-64a1-4c00-a8c2-843060617593 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:07.857987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05babf41-64a1-4c00-a8c2-843060617593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:07.860786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05babf41-64a1-4c00-a8c2-843060617593 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:07.901930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1107/2204] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:07 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:16:07.909356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05babf41-64a1-4c00-a8c2-843060617593 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:07.931305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8358c6201134e52a0ab36146cb470b7', 'request_id': 'req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.961532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8358c6201134e52a0ab36146cb470b7', 'request_id': 'req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:07.987400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8358c6201134e52a0ab36146cb470b7', 'request_id': 'req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:08.000910 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:08.020262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] The token cache is being invalidate because project e8358c6201134e52a0ab36146cb470b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:08.022134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:08.027656 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0b9cebd2-e5d3-4539-938a-e0ea1b531a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:08.031765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1098/2205] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:07 2022] DELETE /identity/v3/projects/e8358c6201134e52a0ab36146cb470b7 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:08.035098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12536c-65f4-48c4-bc1e-7049e52564a2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.036541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12536c-65f4-48c4-bc1e-7049e52564a2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.036903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12536c-65f4-48c4-bc1e-7049e52564a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.065195 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9b12536c-65f4-48c4-bc1e-7049e52564a2 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:08.066725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9b12536c-65f4-48c4-bc1e-7049e52564a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:08.084237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05babf41-64a1-4c00-a8c2-843060617593 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:08.229621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1099/2206] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:08.237866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd8c2-ad61-4ebe-b782-0da2b384af4b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.238174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd8c2-ad61-4ebe-b782-0da2b384af4b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.238531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd8c2-ad61-4ebe-b782-0da2b384af4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.252939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8fba3-37b9-49d3-af96-d7b858a751ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:08.254093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8fba3-37b9-49d3-af96-d7b858a751ba admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.256017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8fba3-37b9-49d3-af96-d7b858a751ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.256596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8fba3-37b9-49d3-af96-d7b858a751ba admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.260162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a8fba3-37b9-49d3-af96-d7b858a751ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:08.266301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a8fba3-37b9-49d3-af96-d7b858a751ba admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:08.274725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-85bfd8c2-ad61-4ebe-b782-0da2b384af4b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:08.276834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-85bfd8c2-ad61-4ebe-b782-0da2b384af4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:08.291485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1100/2207] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:16:08.296444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfd8c2-ad61-4ebe-b782-0da2b384af4b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:08.308115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfd8c2-ad61-4ebe-b782-0da2b384af4b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:08.314872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5eeed-8198-4b0b-897c-97b6b53eb26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:08.315575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5eeed-8198-4b0b-897c-97b6b53eb26a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.315857 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5eeed-8198-4b0b-897c-97b6b53eb26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.316131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5eeed-8198-4b0b-897c-97b6b53eb26a admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.317703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5eeed-8198-4b0b-897c-97b6b53eb26a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:08.318059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5eeed-8198-4b0b-897c-97b6b53eb26a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:08.318326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1108/2208] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:07 2022] POST /identity/v3/users => generated 293 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:08.320204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5eeed-8198-4b0b-897c-97b6b53eb26a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:08.322182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8fba3-37b9-49d3-af96-d7b858a751ba admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:08.324166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-33a8fba3-37b9-49d3-af96-d7b858a751ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:08.342378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1101/2209] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 04 09:16:08.344021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5eeed-8198-4b0b-897c-97b6b53eb26a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:08.353251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8fba3-37b9-49d3-af96-d7b858a751ba admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:08.366843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5eeed-8198-4b0b-897c-97b6b53eb26a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:08.368643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15cb53-e450-4e09-82e4-5493086c8cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:08.369345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15cb53-e450-4e09-82e4-5493086c8cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.369632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15cb53-e450-4e09-82e4-5493086c8cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.369932 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15cb53-e450-4e09-82e4-5493086c8cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.370683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15cb53-e450-4e09-82e4-5493086c8cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:08.372495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15cb53-e450-4e09-82e4-5493086c8cdf admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:08.372608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249aa47-61b7-4537-addf-69a9e1a791bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:08.373400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249aa47-61b7-4537-addf-69a9e1a791bc admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.373698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249aa47-61b7-4537-addf-69a9e1a791bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.374104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249aa47-61b7-4537-addf-69a9e1a791bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.374808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249aa47-61b7-4537-addf-69a9e1a791bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:08.384842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249aa47-61b7-4537-addf-69a9e1a791bc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:08.392625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1109/2210] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:08.403597 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15cb53-e450-4e09-82e4-5493086c8cdf admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:08.416292 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15cb53-e450-4e09-82e4-5493086c8cdf admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:08.421802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db9f45-5f0d-49a6-a73d-fc677bfb793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:08.423017 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db9f45-5f0d-49a6-a73d-fc677bfb793c admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.423572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db9f45-5f0d-49a6-a73d-fc677bfb793c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.423908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db9f45-5f0d-49a6-a73d-fc677bfb793c admin admin] PATH_INFO: `/v3/projects/e49b405054e44158a62083dcd2ece0d7/users/364a7e1a20364a58b1c4669cce7aacee/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.438873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1102/2211] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:08 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:08.439846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249aa47-61b7-4537-addf-69a9e1a791bc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:08.441944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db9f45-5f0d-49a6-a73d-fc677bfb793c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e49b405054e44158a62083dcd2ece0d7, user_id=364a7e1a20364a58b1c4669cce7aacee, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:08.445346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db9f45-5f0d-49a6-a73d-fc677bfb793c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:08.474537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1110/2212] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:08 2022] PUT /identity/v3/projects/e49b405054e44158a62083dcd2ece0d7/users/364a7e1a20364a58b1c4669cce7aacee/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:08.480487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db9f45-5f0d-49a6-a73d-fc677bfb793c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:08.596304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:08.629907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db9f45-5f0d-49a6-a73d-fc677bfb793c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:08.630422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249aa47-61b7-4537-addf-69a9e1a791bc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:08.636362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38a4c6-3fdf-4a41-94e7-8c6676b73016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:08.637387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38a4c6-3fdf-4a41-94e7-8c6676b73016 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.637830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38a4c6-3fdf-4a41-94e7-8c6676b73016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.638288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38a4c6-3fdf-4a41-94e7-8c6676b73016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.639515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38a4c6-3fdf-4a41-94e7-8c6676b73016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:08.641257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38a4c6-3fdf-4a41-94e7-8c6676b73016 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:08.648304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c11ca3-8af8-4ca9-9268-227aba89cd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:08.649797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c11ca3-8af8-4ca9-9268-227aba89cd57 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.650114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c11ca3-8af8-4ca9-9268-227aba89cd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.650432 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c11ca3-8af8-4ca9-9268-227aba89cd57 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.651203 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c11ca3-8af8-4ca9-9268-227aba89cd57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:08.653204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c11ca3-8af8-4ca9-9268-227aba89cd57 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:08.659661 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1111/2213] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:16:08.661037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38a4c6-3fdf-4a41-94e7-8c6676b73016 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:08.663991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7a38a4c6-3fdf-4a41-94e7-8c6676b73016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:08.689917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1103/2214] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:08 2022] POST /identity/v3/users => generated 287 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:08.690763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c11ca3-8af8-4ca9-9268-227aba89cd57 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:08.693500 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38a4c6-3fdf-4a41-94e7-8c6676b73016 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:08.699370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:08.700823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.701229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.702550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] PATH_INFO: `/v3/projects/41f4066e2e4941c482307c56b59bd7bc` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.704165 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f4066e2e4941c482307c56b59bd7bc)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:08.706793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:08.713273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c11ca3-8af8-4ca9-9268-227aba89cd57 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:08.725264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26f79-411d-4466-af88-32ef7a9a39d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:08.726212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26f79-411d-4466-af88-32ef7a9a39d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.726498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26f79-411d-4466-af88-32ef7a9a39d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.726777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26f79-411d-4466-af88-32ef7a9a39d7 admin admin] PATH_INFO: `/v3/projects/e49b405054e44158a62083dcd2ece0d7/users/364a7e1a20364a58b1c4669cce7aacee/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.731550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd26f79-411d-4466-af88-32ef7a9a39d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e49b405054e44158a62083dcd2ece0d7, user_id=364a7e1a20364a58b1c4669cce7aacee, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:08.735079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd26f79-411d-4466-af88-32ef7a9a39d7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:08.759059 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:08.766084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1104/2215] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:08 2022] PUT /identity/v3/projects/e49b405054e44158a62083dcd2ece0d7/users/364a7e1a20364a58b1c4669cce7aacee/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:08.767078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26f79-411d-4466-af88-32ef7a9a39d7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:08.847487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f4066e2e4941c482307c56b59bd7bc', 'request_id': 'req-4e4d7d15-0c09-442d-95e2-a65caa869e91'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:08.871744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f4066e2e4941c482307c56b59bd7bc', 'request_id': 'req-4e4d7d15-0c09-442d-95e2-a65caa869e91'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:08.883696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f4066e2e4941c482307c56b59bd7bc', 'request_id': 'req-4e4d7d15-0c09-442d-95e2-a65caa869e91'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:08.906539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] The token cache is being invalidate because project 41f4066e2e4941c482307c56b59bd7bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:08.907671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:08.911790 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4e4d7d15-0c09-442d-95e2-a65caa869e91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:08.914213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1112/2216] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:08 2022] DELETE /identity/v3/projects/41f4066e2e4941c482307c56b59bd7bc => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:08.919546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243359e-85a1-4747-aa48-524ea0c756a9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:08.919993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243359e-85a1-4747-aa48-524ea0c756a9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:08.920813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243359e-85a1-4747-aa48-524ea0c756a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:08.927094 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26f79-411d-4466-af88-32ef7a9a39d7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:08.966009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4243359e-85a1-4747-aa48-524ea0c756a9 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:08.971681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4243359e-85a1-4747-aa48-524ea0c756a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:09.070721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ede67-685b-4e48-ba4f-a39672163461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.071407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ede67-685b-4e48-ba4f-a39672163461 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.071714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ede67-685b-4e48-ba4f-a39672163461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.072112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ede67-685b-4e48-ba4f-a39672163461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.072866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ede67-685b-4e48-ba4f-a39672163461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.074762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ede67-685b-4e48-ba4f-a39672163461 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.091668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1105/2217] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:09.093339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ede67-685b-4e48-ba4f-a39672163461 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.095689 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bc4ede67-685b-4e48-ba4f-a39672163461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:09.123648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ede67-685b-4e48-ba4f-a39672163461 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.124478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1113/2218] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:09.132991 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121c113-6dcc-477a-bfa2-be64879322d4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.133288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121c113-6dcc-477a-bfa2-be64879322d4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.134781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121c113-6dcc-477a-bfa2-be64879322d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.158544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e970a5-245c-4928-a30a-111a806793fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.162756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e970a5-245c-4928-a30a-111a806793fb admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.163288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e970a5-245c-4928-a30a-111a806793fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.164878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e970a5-245c-4928-a30a-111a806793fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.167101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e970a5-245c-4928-a30a-111a806793fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.169429 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e970a5-245c-4928-a30a-111a806793fb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.173807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0121c113-6dcc-477a-bfa2-be64879322d4 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:09.175107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0121c113-6dcc-477a-bfa2-be64879322d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:09.186382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1106/2219] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:09.188652 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e970a5-245c-4928-a30a-111a806793fb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.194461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1114/2220] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:09.199570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121c113-6dcc-477a-bfa2-be64879322d4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.207581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e970a5-245c-4928-a30a-111a806793fb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.212576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121c113-6dcc-477a-bfa2-be64879322d4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.215570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceda0c6-6c25-48a2-8d70-bd638cf77b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.216493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda0c6-6c25-48a2-8d70-bd638cf77b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.216779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda0c6-6c25-48a2-8d70-bd638cf77b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.217058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda0c6-6c25-48a2-8d70-bd638cf77b5b admin admin] PATH_INFO: `/v3/projects/b7869946a7ac4ac7a15fab3dfe687abd/users/34edc297b377425b92683fc6530e8135/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.222337 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0444a31-d81b-4c15-b969-0cf97a880dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.223229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0444a31-d81b-4c15-b969-0cf97a880dea admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.223521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0444a31-d81b-4c15-b969-0cf97a880dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.223608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0444a31-d81b-4c15-b969-0cf97a880dea admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.224274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0444a31-d81b-4c15-b969-0cf97a880dea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:09.224569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0444a31-d81b-4c15-b969-0cf97a880dea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.226225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0444a31-d81b-4c15-b969-0cf97a880dea admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.257349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceda0c6-6c25-48a2-8d70-bd638cf77b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7869946a7ac4ac7a15fab3dfe687abd, user_id=34edc297b377425b92683fc6530e8135, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.261570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1115/2221] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:16:09.261735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceda0c6-6c25-48a2-8d70-bd638cf77b5b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.267302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25410d3-8045-4cbd-bcae-1e0c953361c9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.267867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25410d3-8045-4cbd-bcae-1e0c953361c9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.268256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25410d3-8045-4cbd-bcae-1e0c953361c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.283066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1107/2222] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:09 2022] PUT /identity/v3/projects/b7869946a7ac4ac7a15fab3dfe687abd/users/34edc297b377425b92683fc6530e8135/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:09.284228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceda0c6-6c25-48a2-8d70-bd638cf77b5b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.322246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f25410d3-8045-4cbd-bcae-1e0c953361c9 None None] MFA Rules not processed for user `5ba79c77fcd54bbcb673a5cba1ed2975`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:09.324462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f25410d3-8045-4cbd-bcae-1e0c953361c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:09.337666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:09.372593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceda0c6-6c25-48a2-8d70-bd638cf77b5b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.377604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d7128-6fbd-4e32-b1fe-f90dc4dcb0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.378380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d7128-6fbd-4e32-b1fe-f90dc4dcb0e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.378668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d7128-6fbd-4e32-b1fe-f90dc4dcb0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.378948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d7128-6fbd-4e32-b1fe-f90dc4dcb0e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.379659 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d7128-6fbd-4e32-b1fe-f90dc4dcb0e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.381706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d7128-6fbd-4e32-b1fe-f90dc4dcb0e6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.418219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1116/2223] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:09.420572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3cc5a-628b-4f30-b8be-d3439d6e58b2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.421089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3cc5a-628b-4f30-b8be-d3439d6e58b2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.421636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3cc5a-628b-4f30-b8be-d3439d6e58b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.427399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1108/2224] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:09 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:09.428981 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d7128-6fbd-4e32-b1fe-f90dc4dcb0e6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.433415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c02d7128-6fbd-4e32-b1fe-f90dc4dcb0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:09.446570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-99b3cc5a-628b-4f30-b8be-d3439d6e58b2 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:09.448586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-99b3cc5a-628b-4f30-b8be-d3439d6e58b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:09.475919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:09.557390 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1117/2225] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:09.563225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3cc5a-628b-4f30-b8be-d3439d6e58b2 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.583145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3cc5a-628b-4f30-b8be-d3439d6e58b2 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.591525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d7128-6fbd-4e32-b1fe-f90dc4dcb0e6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.595181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7414ef0-63f7-4d7e-a0ab-fb0f272d4717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.596026 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7414ef0-63f7-4d7e-a0ab-fb0f272d4717 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.596228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7414ef0-63f7-4d7e-a0ab-fb0f272d4717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.596633 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7414ef0-63f7-4d7e-a0ab-fb0f272d4717 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.597353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7414ef0-63f7-4d7e-a0ab-fb0f272d4717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.601457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7414ef0-63f7-4d7e-a0ab-fb0f272d4717 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.602012 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79988391-edfe-4628-81ef-42088fc263a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.603088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79988391-edfe-4628-81ef-42088fc263a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.603570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79988391-edfe-4628-81ef-42088fc263a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.604037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79988391-edfe-4628-81ef-42088fc263a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.604946 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79988391-edfe-4628-81ef-42088fc263a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.607265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79988391-edfe-4628-81ef-42088fc263a5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.638697 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1118/2226] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:09 2022] POST /identity/v3/users => generated 287 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:09.639605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7414ef0-63f7-4d7e-a0ab-fb0f272d4717 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.641091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1109/2227] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:09 2022] POST /identity/v3/projects => generated 373 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:09.641304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b7414ef0-63f7-4d7e-a0ab-fb0f272d4717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:09.646034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79988391-edfe-4628-81ef-42088fc263a5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.724221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7414ef0-63f7-4d7e-a0ab-fb0f272d4717 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.742554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79988391-edfe-4628-81ef-42088fc263a5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.750392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed7d9-22e1-476e-a901-c37b1241cf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.751581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed7d9-22e1-476e-a901-c37b1241cf53 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.752155 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed7d9-22e1-476e-a901-c37b1241cf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.753030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed7d9-22e1-476e-a901-c37b1241cf53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.754185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fed7d9-22e1-476e-a901-c37b1241cf53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.756421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fed7d9-22e1-476e-a901-c37b1241cf53 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.772014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1110/2228] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:09.773390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed7d9-22e1-476e-a901-c37b1241cf53 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.785725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed7d9-22e1-476e-a901-c37b1241cf53 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.795064 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570519e-5510-4602-9fab-0dc02121b4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.795767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570519e-5510-4602-9fab-0dc02121b4ae admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.796210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570519e-5510-4602-9fab-0dc02121b4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.796491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570519e-5510-4602-9fab-0dc02121b4ae admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.797261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570519e-5510-4602-9fab-0dc02121b4ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.797589 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a5111-b683-44a6-a66e-fff3eb6ddc5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.798767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a5111-b683-44a6-a66e-fff3eb6ddc5f service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.799847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a5111-b683-44a6-a66e-fff3eb6ddc5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.799963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570519e-5510-4602-9fab-0dc02121b4ae admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.800447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a5111-b683-44a6-a66e-fff3eb6ddc5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.816078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a5111-b683-44a6-a66e-fff3eb6ddc5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.820238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a5111-b683-44a6-a66e-fff3eb6ddc5f service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.836033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1111/2229] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:09 2022] POST /identity/v3/users => generated 293 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:09.836033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570519e-5510-4602-9fab-0dc02121b4ae admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.839046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1119/2230] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:09.840057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a5111-b683-44a6-a66e-fff3eb6ddc5f service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.850728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570519e-5510-4602-9fab-0dc02121b4ae admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.856097 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a5111-b683-44a6-a66e-fff3eb6ddc5f service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.862295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c069b31-061a-429b-8987-e23644a11f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.864435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c069b31-061a-429b-8987-e23644a11f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.865503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c069b31-061a-429b-8987-e23644a11f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.866164 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c069b31-061a-429b-8987-e23644a11f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.867241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c069b31-061a-429b-8987-e23644a11f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.869320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c069b31-061a-429b-8987-e23644a11f31 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.871130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16488d-6ff7-48cf-8925-18fcd1cafda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.871823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16488d-6ff7-48cf-8925-18fcd1cafda8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.872107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16488d-6ff7-48cf-8925-18fcd1cafda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.872385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16488d-6ff7-48cf-8925-18fcd1cafda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.877090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16488d-6ff7-48cf-8925-18fcd1cafda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.879711 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16488d-6ff7-48cf-8925-18fcd1cafda8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.895492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1120/2231] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:09.897826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1112/2232] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:09.899447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16488d-6ff7-48cf-8925-18fcd1cafda8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.902000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c069b31-061a-429b-8987-e23644a11f31 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:09.914995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c069b31-061a-429b-8987-e23644a11f31 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.923611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b727b-8350-4466-b2da-c64f28eea85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.923771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16488d-6ff7-48cf-8925-18fcd1cafda8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:09.925348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b727b-8350-4466-b2da-c64f28eea85d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.925712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b727b-8350-4466-b2da-c64f28eea85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.926066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b727b-8350-4466-b2da-c64f28eea85d admin admin] PATH_INFO: `/v3/projects/711c911c986f4d15906497db3cb3f50e/users/dbfd2c9a631340a9b8fea923d2db1c47/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.929104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:09.934362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a66b59-c4c8-4f8b-bd50-8fb4f71ca820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:09.936524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a66b59-c4c8-4f8b-bd50-8fb4f71ca820 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:09.936524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a66b59-c4c8-4f8b-bd50-8fb4f71ca820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:09.936524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a66b59-c4c8-4f8b-bd50-8fb4f71ca820 admin admin] PATH_INFO: `/v3/projects/abfe2a72af8d4f658c560f45f82d0421/users/b89afc9411be4575b1b94d3c1e7b1eb2/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:09.949209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b727b-8350-4466-b2da-c64f28eea85d admin admin] RBAC: Authorizing `identity:create_grant(project_id=711c911c986f4d15906497db3cb3f50e, user_id=dbfd2c9a631340a9b8fea923d2db1c47, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.954421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b727b-8350-4466-b2da-c64f28eea85d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.961803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a66b59-c4c8-4f8b-bd50-8fb4f71ca820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abfe2a72af8d4f658c560f45f82d0421, user_id=b89afc9411be4575b1b94d3c1e7b1eb2, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:09.968219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a66b59-c4c8-4f8b-bd50-8fb4f71ca820 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:09.994445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1113/2233] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:09 2022] PUT /identity/v3/projects/711c911c986f4d15906497db3cb3f50e/users/dbfd2c9a631340a9b8fea923d2db1c47/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:09.997789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1121/2234] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:09 2022] PUT /identity/v3/projects/abfe2a72af8d4f658c560f45f82d0421/users/b89afc9411be4575b1b94d3c1e7b1eb2/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:16:10.008105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b727b-8350-4466-b2da-c64f28eea85d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:10.011575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62df2a-9ffd-4b9c-a91f-03892761cb86 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:10.012054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62df2a-9ffd-4b9c-a91f-03892761cb86 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:10.012405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62df2a-9ffd-4b9c-a91f-03892761cb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:10.087850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4d62df2a-9ffd-4b9c-a91f-03892761cb86 None None] MFA Rules not processed for user `dbfd2c9a631340a9b8fea923d2db1c47`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:10.089147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4d62df2a-9ffd-4b9c-a91f-03892761cb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:10.122879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:10.129205 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b727b-8350-4466-b2da-c64f28eea85d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:10.213666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1122/2235] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:10.231383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62df2a-9ffd-4b9c-a91f-03892761cb86 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:10.233803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4d62df2a-9ffd-4b9c-a91f-03892761cb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:10.331477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62df2a-9ffd-4b9c-a91f-03892761cb86 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:10.332697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bd0e0-21e3-4004-9c84-ea243a0f570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:10.333813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bd0e0-21e3-4004-9c84-ea243a0f570c admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:10.334365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bd0e0-21e3-4004-9c84-ea243a0f570c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:10.335657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05df7a5-d9d9-4e92-ad8d-6d62b26333aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:10.336406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05df7a5-d9d9-4e92-ad8d-6d62b26333aa service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:10.336634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05df7a5-d9d9-4e92-ad8d-6d62b26333aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:10.336922 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05df7a5-d9d9-4e92-ad8d-6d62b26333aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:10.338210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bd0e0-21e3-4004-9c84-ea243a0f570c admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:10.339340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242bd0e0-21e3-4004-9c84-ea243a0f570c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:10.342343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242bd0e0-21e3-4004-9c84-ea243a0f570c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:10.353221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05df7a5-d9d9-4e92-ad8d-6d62b26333aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:10.357912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05df7a5-d9d9-4e92-ad8d-6d62b26333aa service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:10.378504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1114/2236] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:10.380406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc974e-7812-47ba-85d3-950dcde2cda5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:10.380886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc974e-7812-47ba-85d3-950dcde2cda5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:10.381259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc974e-7812-47ba-85d3-950dcde2cda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:10.403718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1123/2237] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:16:10.420587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05df7a5-d9d9-4e92-ad8d-6d62b26333aa service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:10.457765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05df7a5-d9d9-4e92-ad8d-6d62b26333aa service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:10.460884 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-efbc974e-7812-47ba-85d3-950dcde2cda5 None None] MFA Rules not processed for user `364a7e1a20364a58b1c4669cce7aacee`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:10.462181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-efbc974e-7812-47ba-85d3-950dcde2cda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:10.473940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c1950-81ae-4082-a5fb-0b7a0098f84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:10.474876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c1950-81ae-4082-a5fb-0b7a0098f84a admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:10.475163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c1950-81ae-4082-a5fb-0b7a0098f84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:10.475447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c1950-81ae-4082-a5fb-0b7a0098f84a admin admin] PATH_INFO: `/v3/projects/abfe2a72af8d4f658c560f45f82d0421/users/b89afc9411be4575b1b94d3c1e7b1eb2/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:10.481482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c1950-81ae-4082-a5fb-0b7a0098f84a admin admin] RBAC: Authorizing `identity:create_grant(project_id=abfe2a72af8d4f658c560f45f82d0421, user_id=b89afc9411be4575b1b94d3c1e7b1eb2, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:10.484487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c1950-81ae-4082-a5fb-0b7a0098f84a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:10.553998 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1124/2238] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:10 2022] PUT /identity/v3/projects/abfe2a72af8d4f658c560f45f82d0421/users/b89afc9411be4575b1b94d3c1e7b1eb2/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:10.558936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c1950-81ae-4082-a5fb-0b7a0098f84a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:10.675612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1115/2239] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:10.699535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c1950-81ae-4082-a5fb-0b7a0098f84a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:10.704726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c70ef-7bce-4a25-ade1-d285846a82e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:10.705715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c70ef-7bce-4a25-ade1-d285846a82e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:10.706361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c70ef-7bce-4a25-ade1-d285846a82e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:10.706464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c70ef-7bce-4a25-ade1-d285846a82e0 admin admin] PATH_INFO: `/v3/users/dbfd2c9a631340a9b8fea923d2db1c47` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:10.708153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c70ef-7bce-4a25-ade1-d285846a82e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbfd2c9a631340a9b8fea923d2db1c47)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:10.710894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c70ef-7bce-4a25-ade1-d285846a82e0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:10.722870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc974e-7812-47ba-85d3-950dcde2cda5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:10.724940 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-efbc974e-7812-47ba-85d3-950dcde2cda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:10.745767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc974e-7812-47ba-85d3-950dcde2cda5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:10.758580 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:10.812784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ba6c70ef-7bce-4a25-ade1-d285846a82e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbfd2c9a631340a9b8fea923d2db1c47'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:10.814079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ba6c70ef-7bce-4a25-ade1-d285846a82e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbfd2c9a631340a9b8fea923d2db1c47'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:10.820680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:10.836399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ba6c70ef-7bce-4a25-ade1-d285846a82e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbfd2c9a631340a9b8fea923d2db1c47'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:10.839174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193001db-45c1-4e3c-b37f-20b8868779aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:10.840717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193001db-45c1-4e3c-b37f-20b8868779aa service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:10.842607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193001db-45c1-4e3c-b37f-20b8868779aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:10.843059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193001db-45c1-4e3c-b37f-20b8868779aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:10.846194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-193001db-45c1-4e3c-b37f-20b8868779aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:10.851766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ba6c70ef-7bce-4a25-ade1-d285846a82e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbfd2c9a631340a9b8fea923d2db1c47'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:10.853618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ba6c70ef-7bce-4a25-ade1-d285846a82e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbfd2c9a631340a9b8fea923d2db1c47'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:10.901164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1125/2240] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:10 2022] DELETE /identity/v3/users/dbfd2c9a631340a9b8fea923d2db1c47 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:10.906566 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eed07-725e-4a82-9c57-2e126d6b2631 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:10.906849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eed07-725e-4a82-9c57-2e126d6b2631 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:10.907317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eed07-725e-4a82-9c57-2e126d6b2631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:10.941274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-351eed07-725e-4a82-9c57-2e126d6b2631 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:10.941992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193001db-45c1-4e3c-b37f-20b8868779aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:10.942763 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-351eed07-725e-4a82-9c57-2e126d6b2631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:10.943674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193001db-45c1-4e3c-b37f-20b8868779aa service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:10.960973 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1116/2241] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:16:11.090388 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1126/2242] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:11.108902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193001db-45c1-4e3c-b37f-20b8868779aa service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:11.110125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-193001db-45c1-4e3c-b37f-20b8868779aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:11.261844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193001db-45c1-4e3c-b37f-20b8868779aa service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:11.386429 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7f5d5-6726-4a49-9730-c37824d5c485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:11.388135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7f5d5-6726-4a49-9730-c37824d5c485 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:11.388490 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7f5d5-6726-4a49-9730-c37824d5c485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:11.389029 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7f5d5-6726-4a49-9730-c37824d5c485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:11.428178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef7f5d5-6726-4a49-9730-c37824d5c485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:11.430063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef7f5d5-6726-4a49-9730-c37824d5c485 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:11.468643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1117/2243] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:11.506354 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6239a-84cc-4877-bdb3-ce9e5862dda0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:11.506779 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6239a-84cc-4877-bdb3-ce9e5862dda0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:11.507153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6239a-84cc-4877-bdb3-ce9e5862dda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:11.596284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-06d6239a-84cc-4877-bdb3-ce9e5862dda0 None None] MFA Rules not processed for user `34edc297b377425b92683fc6530e8135`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:11.597530 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-06d6239a-84cc-4877-bdb3-ce9e5862dda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:11.714081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1127/2244] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:11.728015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7f5d5-6726-4a49-9730-c37824d5c485 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:11.734676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1ef7f5d5-6726-4a49-9730-c37824d5c485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:11.779676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:11.817707 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7f5d5-6726-4a49-9730-c37824d5c485 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:11.823107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de59fd5-1b60-4798-8cf5-e0c3c837aab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:11.823886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de59fd5-1b60-4798-8cf5-e0c3c837aab9 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:11.824182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de59fd5-1b60-4798-8cf5-e0c3c837aab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:11.824470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de59fd5-1b60-4798-8cf5-e0c3c837aab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:11.838097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de59fd5-1b60-4798-8cf5-e0c3c837aab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:11.840963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de59fd5-1b60-4798-8cf5-e0c3c837aab9 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:11.865009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1118/2245] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:12.224119 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6239a-84cc-4877-bdb3-ce9e5862dda0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:12.226471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-06d6239a-84cc-4877-bdb3-ce9e5862dda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:12.256616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6239a-84cc-4877-bdb3-ce9e5862dda0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:12.266293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:12.266845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:12.267137 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:12.267810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] PATH_INFO: `/v3/projects/711c911c986f4d15906497db3cb3f50e` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:12.276753 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=711c911c986f4d15906497db3cb3f50e)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:12.278811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:12.377519 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711c911c986f4d15906497db3cb3f50e', 'request_id': 'req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:12.383735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:12.399024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711c911c986f4d15906497db3cb3f50e', 'request_id': 'req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:12.413850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711c911c986f4d15906497db3cb3f50e', 'request_id': 'req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:12.427786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] The token cache is being invalidate because project 711c911c986f4d15906497db3cb3f50e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:12.428419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:12.429396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9603bd69-31a0-4893-98a8-3ec7a4b1ff7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:12.431304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1128/2246] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:12 2022] DELETE /identity/v3/projects/711c911c986f4d15906497db3cb3f50e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:12.436350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e46e-5ad0-47f9-8b91-ef258a7c2246 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:12.436710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e46e-5ad0-47f9-8b91-ef258a7c2246 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:12.436925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e46e-5ad0-47f9-8b91-ef258a7c2246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:12.478095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0543e46e-5ad0-47f9-8b91-ef258a7c2246 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:12.479537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0543e46e-5ad0-47f9-8b91-ef258a7c2246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:12.575018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1119/2247] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:12.582941 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd47ee-9cff-4d5f-8aba-c4c248b8c608 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:12.583448 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd47ee-9cff-4d5f-8aba-c4c248b8c608 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:12.583816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd47ee-9cff-4d5f-8aba-c4c248b8c608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:12.607752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c4fd47ee-9cff-4d5f-8aba-c4c248b8c608 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:12.609524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c4fd47ee-9cff-4d5f-8aba-c4c248b8c608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:12.629608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1129/2248] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:12.633588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543e46e-5ad0-47f9-8b91-ef258a7c2246 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:12.645224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543e46e-5ad0-47f9-8b91-ef258a7c2246 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:12.651601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28921b69-7356-43c7-953a-fbff9c187212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:12.652715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28921b69-7356-43c7-953a-fbff9c187212 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:12.652715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28921b69-7356-43c7-953a-fbff9c187212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:12.652715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28921b69-7356-43c7-953a-fbff9c187212 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:12.654998 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28921b69-7356-43c7-953a-fbff9c187212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:12.655482 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28921b69-7356-43c7-953a-fbff9c187212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:12.657733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28921b69-7356-43c7-953a-fbff9c187212 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:12.682708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd47ee-9cff-4d5f-8aba-c4c248b8c608 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:12.692358 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd47ee-9cff-4d5f-8aba-c4c248b8c608 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:12.693438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1120/2249] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:16:12.699711 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c7d1d-0498-4036-871d-91af6d9939c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:12.700928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7d1d-0498-4036-871d-91af6d9939c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:12.700928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7d1d-0498-4036-871d-91af6d9939c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:12.701392 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7d1d-0498-4036-871d-91af6d9939c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:12.703322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c7d1d-0498-4036-871d-91af6d9939c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:12.707664 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c7d1d-0498-4036-871d-91af6d9939c2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:12.758551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1130/2250] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:12 2022] POST /identity/v3/projects => generated 399 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:16:12.760343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28921b69-7356-43c7-953a-fbff9c187212 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:12.775672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c7d1d-0498-4036-871d-91af6d9939c2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:12.777838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28921b69-7356-43c7-953a-fbff9c187212 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:12.778044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-747c7d1d-0498-4036-871d-91af6d9939c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:12.877641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c7d1d-0498-4036-871d-91af6d9939c2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:12.888367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38265302-4053-4938-a924-2d303e58e733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:12.889019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38265302-4053-4938-a924-2d303e58e733 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:12.889301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38265302-4053-4938-a924-2d303e58e733 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:12.889583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38265302-4053-4938-a924-2d303e58e733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:12.891934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-38265302-4053-4938-a924-2d303e58e733 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:12.941288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d96f5-fc11-448f-9241-57d5558119a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:12.944775 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d96f5-fc11-448f-9241-57d5558119a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:12.945106 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d96f5-fc11-448f-9241-57d5558119a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:12.945380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d96f5-fc11-448f-9241-57d5558119a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:12.946940 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d96f5-fc11-448f-9241-57d5558119a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:12.959231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d96f5-fc11-448f-9241-57d5558119a7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:13.034173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1121/2251] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:12 2022] POST /identity/v3/users => generated 300 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:13.038089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d96f5-fc11-448f-9241-57d5558119a7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:13.067220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38265302-4053-4938-a924-2d303e58e733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:13.068794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38265302-4053-4938-a924-2d303e58e733 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:13.068910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d96f5-fc11-448f-9241-57d5558119a7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:13.088726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1131/2252] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:16:13.090964 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea6a30-ce08-4147-af2a-efa38dca28d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:13.091751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea6a30-ce08-4147-af2a-efa38dca28d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:13.092234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea6a30-ce08-4147-af2a-efa38dca28d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:13.092574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea6a30-ce08-4147-af2a-efa38dca28d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:13.093701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea6a30-ce08-4147-af2a-efa38dca28d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:13.096576 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea6a30-ce08-4147-af2a-efa38dca28d7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:13.124813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1122/2253] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:16:13.125857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38265302-4053-4938-a924-2d303e58e733 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:13.144421 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38265302-4053-4938-a924-2d303e58e733 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:13.170207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07fc016-9583-435c-b6b8-89013f26dd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:13.171451 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fc016-9583-435c-b6b8-89013f26dd10 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:13.172076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fc016-9583-435c-b6b8-89013f26dd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:13.175208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fc016-9583-435c-b6b8-89013f26dd10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:13.178161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fc016-9583-435c-b6b8-89013f26dd10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:13.184124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fc016-9583-435c-b6b8-89013f26dd10 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:13.210543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1132/2254] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:16:13.214620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea6a30-ce08-4147-af2a-efa38dca28d7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:13.233127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea6a30-ce08-4147-af2a-efa38dca28d7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:13.241733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d31cd5-fc17-42c1-b861-148dadddf0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:13.242953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d31cd5-fc17-42c1-b861-148dadddf0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:13.243486 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d31cd5-fc17-42c1-b861-148dadddf0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:13.243861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d31cd5-fc17-42c1-b861-148dadddf0eb admin admin] PATH_INFO: `/v3/projects/68d2a8094f8446a0a86e05c76bb6dc87/users/b4622e92dbf747049e1f829ca2195c88/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:13.267480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d31cd5-fc17-42c1-b861-148dadddf0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d2a8094f8446a0a86e05c76bb6dc87, user_id=b4622e92dbf747049e1f829ca2195c88, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:13.271549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d31cd5-fc17-42c1-b861-148dadddf0eb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:13.314816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1123/2255] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:13 2022] PUT /identity/v3/projects/68d2a8094f8446a0a86e05c76bb6dc87/users/b4622e92dbf747049e1f829ca2195c88/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:13.316584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197b58-1890-4305-9e28-710902e448e1 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:13.317060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197b58-1890-4305-9e28-710902e448e1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:13.317146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197b58-1890-4305-9e28-710902e448e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:13.374338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f2197b58-1890-4305-9e28-710902e448e1 None None] MFA Rules not processed for user `b4622e92dbf747049e1f829ca2195c88`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:13.380362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f2197b58-1890-4305-9e28-710902e448e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:13.505287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14cda2-0b0e-4989-9ec3-8faf3951949c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:13.505819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14cda2-0b0e-4989-9ec3-8faf3951949c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:13.506895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14cda2-0b0e-4989-9ec3-8faf3951949c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:13.526302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:13.539658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2b14cda2-0b0e-4989-9ec3-8faf3951949c None None] MFA Rules not processed for user `34edc297b377425b92683fc6530e8135`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:13.542657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2b14cda2-0b0e-4989-9ec3-8faf3951949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:13.549992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:13.582695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1133/2256] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:16:13.596588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2197b58-1890-4305-9e28-710902e448e1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:13.692567 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1124/2257] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:13.704384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2197b58-1890-4305-9e28-710902e448e1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:13.708703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef1bfa-0555-4278-af91-ca13b3b4e77c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:13.710787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef1bfa-0555-4278-af91-ca13b3b4e77c service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:13.711076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef1bfa-0555-4278-af91-ca13b3b4e77c service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:13.711404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef1bfa-0555-4278-af91-ca13b3b4e77c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:13.722483 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef1bfa-0555-4278-af91-ca13b3b4e77c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:13.724170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef1bfa-0555-4278-af91-ca13b3b4e77c service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:13.754851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1134/2258] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:15.626574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14cda2-0b0e-4989-9ec3-8faf3951949c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:15.628915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2b14cda2-0b0e-4989-9ec3-8faf3951949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:15.832752 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14cda2-0b0e-4989-9ec3-8faf3951949c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:15.841669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:15.842928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:15.843410 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:15.843837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] PATH_INFO: `/v3/users/03ec9afc1d8f47f69e0b9656dbd4725c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:15.847507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ec9afc1d8f47f69e0b9656dbd4725c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:15.852350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:15.963493 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ec9afc1d8f47f69e0b9656dbd4725c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:15.964370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ec9afc1d8f47f69e0b9656dbd4725c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:15.965370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ec9afc1d8f47f69e0b9656dbd4725c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:15.985698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ec9afc1d8f47f69e0b9656dbd4725c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:16.010293 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ec9afc1d8f47f69e0b9656dbd4725c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:16.036055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1125/2259] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:15 2022] DELETE /identity/v3/users/03ec9afc1d8f47f69e0b9656dbd4725c => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:16.041297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0295c-7ad2-4586-a4a8-f4789cfa7948 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:16.041864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0295c-7ad2-4586-a4a8-f4789cfa7948 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:16.042204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0295c-7ad2-4586-a4a8-f4789cfa7948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:16.090230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d3b0295c-7ad2-4586-a4a8-f4789cfa7948 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:16.091964 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d3b0295c-7ad2-4586-a4a8-f4789cfa7948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:16.241565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1135/2260] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:16.262956 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:16.264565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:16.380453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:16.433067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07cc1e-a6ab-4b88-9efd-9caafa7cf320 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:16.448004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7d65c-39b9-40c8-a0b5-ccc9720b15a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:16.450636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7d65c-39b9-40c8-a0b5-ccc9720b15a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:16.451000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7d65c-39b9-40c8-a0b5-ccc9720b15a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:16.452755 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7d65c-39b9-40c8-a0b5-ccc9720b15a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:16.491679 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7d65c-39b9-40c8-a0b5-ccc9720b15a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:16.500710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7d65c-39b9-40c8-a0b5-ccc9720b15a0 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:16.545711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1126/2261] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:17.408793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0295c-7ad2-4586-a4a8-f4789cfa7948 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:17.411278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d3b0295c-7ad2-4586-a4a8-f4789cfa7948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:17.434955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0295c-7ad2-4586-a4a8-f4789cfa7948 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:17.441851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:17.442779 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:17.443058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:17.446162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] PATH_INFO: `/v3/projects/fbaa9b9c6e0b4180bb267874bbbb05ad` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:17.454400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbaa9b9c6e0b4180bb267874bbbb05ad)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:17.456626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:17.584653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbaa9b9c6e0b4180bb267874bbbb05ad', 'request_id': 'req-9a803e82-a63e-43e1-aad0-967a771f229f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:17.597541 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbaa9b9c6e0b4180bb267874bbbb05ad', 'request_id': 'req-9a803e82-a63e-43e1-aad0-967a771f229f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:17.605278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbaa9b9c6e0b4180bb267874bbbb05ad', 'request_id': 'req-9a803e82-a63e-43e1-aad0-967a771f229f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:17.620289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] The token cache is being invalidate because project fbaa9b9c6e0b4180bb267874bbbb05ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:17.620827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:17.621636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-9a803e82-a63e-43e1-aad0-967a771f229f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:17.625762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1136/2262] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:17 2022] DELETE /identity/v3/projects/fbaa9b9c6e0b4180bb267874bbbb05ad => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:17.630495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128b4db-cea7-4cf8-b860-4ec07c2dec9b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:17.631021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128b4db-cea7-4cf8-b860-4ec07c2dec9b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:17.631263 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128b4db-cea7-4cf8-b860-4ec07c2dec9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:17.677860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c128b4db-cea7-4cf8-b860-4ec07c2dec9b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:17.679925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c128b4db-cea7-4cf8-b860-4ec07c2dec9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:17.811162 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1127/2263] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:17.812880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4acec-279b-4a48-816e-38cfbad9bca8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:17.813166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4acec-279b-4a48-816e-38cfbad9bca8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:17.813441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4acec-279b-4a48-816e-38cfbad9bca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:17.833794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:17.849097 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-14e4acec-279b-4a48-816e-38cfbad9bca8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:17.861032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-14e4acec-279b-4a48-816e-38cfbad9bca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:17.881271 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1137/2264] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:17.883095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128b4db-cea7-4cf8-b860-4ec07c2dec9b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:17.900261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128b4db-cea7-4cf8-b860-4ec07c2dec9b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:17.907467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b73c0-f629-4238-a749-23d936650e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:17.908386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b73c0-f629-4238-a749-23d936650e09 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:17.909149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b73c0-f629-4238-a749-23d936650e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:17.909531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b73c0-f629-4238-a749-23d936650e09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:17.910695 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b73c0-f629-4238-a749-23d936650e09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:17.911362 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b73c0-f629-4238-a749-23d936650e09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:17.912943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b73c0-f629-4238-a749-23d936650e09 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:17.937764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1128/2265] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:16:17.939070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4acec-279b-4a48-816e-38cfbad9bca8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:17.949399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4acec-279b-4a48-816e-38cfbad9bca8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:17.955562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd5b2b-13c2-4fae-ba0f-32a4b1e396b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:17.956222 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd5b2b-13c2-4fae-ba0f-32a4b1e396b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:17.956814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd5b2b-13c2-4fae-ba0f-32a4b1e396b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:17.957202 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd5b2b-13c2-4fae-ba0f-32a4b1e396b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:17.958056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd5b2b-13c2-4fae-ba0f-32a4b1e396b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:17.960056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd5b2b-13c2-4fae-ba0f-32a4b1e396b4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:18.001414 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1138/2266] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:17 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:18.003302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b73c0-f629-4238-a749-23d936650e09 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:18.016144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b73c0-f629-4238-a749-23d936650e09 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:18.206743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b76001-5fe3-4fb6-a6e4-09156acf6e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:18.207892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b76001-5fe3-4fb6-a6e4-09156acf6e5a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:18.208381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b76001-5fe3-4fb6-a6e4-09156acf6e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:18.212042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b76001-5fe3-4fb6-a6e4-09156acf6e5a admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:18.212042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b76001-5fe3-4fb6-a6e4-09156acf6e5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:18.216163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b76001-5fe3-4fb6-a6e4-09156acf6e5a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:18.266536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1129/2267] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:18 2022] POST /identity/v3/users => generated 283 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:18.272476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd5b2b-13c2-4fae-ba0f-32a4b1e396b4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:18.307167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd5b2b-13c2-4fae-ba0f-32a4b1e396b4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:18.324050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8465c5c-409a-4d2f-b2a9-e5b21cae7b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:18.324843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8465c5c-409a-4d2f-b2a9-e5b21cae7b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:18.325318 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8465c5c-409a-4d2f-b2a9-e5b21cae7b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:18.325789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8465c5c-409a-4d2f-b2a9-e5b21cae7b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:18.327349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8465c5c-409a-4d2f-b2a9-e5b21cae7b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:18.330460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8465c5c-409a-4d2f-b2a9-e5b21cae7b7a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:18.344081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1139/2268] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:18.348580 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b76001-5fe3-4fb6-a6e4-09156acf6e5a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:18.381182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b76001-5fe3-4fb6-a6e4-09156acf6e5a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:18.399319 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020529e-e0ed-4dcd-9dd5-fa413cb00efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:18.400027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020529e-e0ed-4dcd-9dd5-fa413cb00efd admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:18.400305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020529e-e0ed-4dcd-9dd5-fa413cb00efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:18.404394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020529e-e0ed-4dcd-9dd5-fa413cb00efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:18.405045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020529e-e0ed-4dcd-9dd5-fa413cb00efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:18.407070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020529e-e0ed-4dcd-9dd5-fa413cb00efd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:18.428091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1130/2269] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:18.432182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8465c5c-409a-4d2f-b2a9-e5b21cae7b7a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:18.448305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8465c5c-409a-4d2f-b2a9-e5b21cae7b7a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:18.457550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9a1c4-6e56-4af1-8c9b-dfdba41a8e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:18.459051 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9a1c4-6e56-4af1-8c9b-dfdba41a8e11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:18.459328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9a1c4-6e56-4af1-8c9b-dfdba41a8e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:18.459766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9a1c4-6e56-4af1-8c9b-dfdba41a8e11 admin admin] PATH_INFO: `/v3/projects/8ae3c1c65e6f4254958b2ea1b031a584/users/f58b8386817145f4a8d5d963444db6a2/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:18.497827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9a1c4-6e56-4af1-8c9b-dfdba41a8e11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae3c1c65e6f4254958b2ea1b031a584, user_id=f58b8386817145f4a8d5d963444db6a2, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:18.501797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9a1c4-6e56-4af1-8c9b-dfdba41a8e11 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:18.517285 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2730d-3053-468f-a1ce-feb4f3cf8f01 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:18.517824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2730d-3053-468f-a1ce-feb4f3cf8f01 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:18.518303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2730d-3053-468f-a1ce-feb4f3cf8f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:18.541470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1140/2270] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:18 2022] PUT /identity/v3/projects/8ae3c1c65e6f4254958b2ea1b031a584/users/f58b8386817145f4a8d5d963444db6a2/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:16:18.555598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86287e71-be6d-4263-b6cb-bfc688459fbf None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:18.556156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86287e71-be6d-4263-b6cb-bfc688459fbf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:18.556585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86287e71-be6d-4263-b6cb-bfc688459fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:18.576440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-36c2730d-3053-468f-a1ce-feb4f3cf8f01 None None] MFA Rules not processed for user `b89afc9411be4575b1b94d3c1e7b1eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:18.577688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-36c2730d-3053-468f-a1ce-feb4f3cf8f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:18.606295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-86287e71-be6d-4263-b6cb-bfc688459fbf None None] MFA Rules not processed for user `f58b8386817145f4a8d5d963444db6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:18.608088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-86287e71-be6d-4263-b6cb-bfc688459fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:18.634093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:18.715728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1141/2271] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:18.729268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1131/2272] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:18.729494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86287e71-be6d-4263-b6cb-bfc688459fbf None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:18.731944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-86287e71-be6d-4263-b6cb-bfc688459fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:18.736431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2730d-3053-468f-a1ce-feb4f3cf8f01 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:18.737788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-36c2730d-3053-468f-a1ce-feb4f3cf8f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:18.822848 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86287e71-be6d-4263-b6cb-bfc688459fbf None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:18.829030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ccc9a-6c17-4839-a164-15d0e23db378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:18.829977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ccc9a-6c17-4839-a164-15d0e23db378 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:18.830399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ccc9a-6c17-4839-a164-15d0e23db378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:18.830761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ccc9a-6c17-4839-a164-15d0e23db378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:18.842308 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ccc9a-6c17-4839-a164-15d0e23db378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:18.844037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ccc9a-6c17-4839-a164-15d0e23db378 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:18.858627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1142/2273] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:18.873348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2730d-3053-468f-a1ce-feb4f3cf8f01 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:18.879552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaf789-7ddf-4859-9716-ed94a20f784d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:18.880324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaf789-7ddf-4859-9716-ed94a20f784d service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:18.880705 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaf789-7ddf-4859-9716-ed94a20f784d service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:18.881081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaf789-7ddf-4859-9716-ed94a20f784d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:18.883238 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8ceaf789-7ddf-4859-9716-ed94a20f784d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:18.975500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaf789-7ddf-4859-9716-ed94a20f784d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:18.989954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaf789-7ddf-4859-9716-ed94a20f784d service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:19.016129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1132/2274] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:16:21.473037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfd942-882d-482d-a114-a4a2719c40d1 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:21.474776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfd942-882d-482d-a114-a4a2719c40d1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:21.475279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfd942-882d-482d-a114-a4a2719c40d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:21.512329 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-70dfd942-882d-482d-a114-a4a2719c40d1 None None] MFA Rules not processed for user `5ba79c77fcd54bbcb673a5cba1ed2975`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:21.517860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-70dfd942-882d-482d-a114-a4a2719c40d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:21.536193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1143/2275] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:22.111051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaf789-7ddf-4859-9716-ed94a20f784d service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:22.114516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8ceaf789-7ddf-4859-9716-ed94a20f784d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:22.273729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaf789-7ddf-4859-9716-ed94a20f784d service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:22.280885 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:22.282254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:22.282632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:22.283002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] PATH_INFO: `/v3/users/b4622e92dbf747049e1f829ca2195c88` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:22.284854 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4622e92dbf747049e1f829ca2195c88)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:22.286999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:22.294614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:22.393610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4622e92dbf747049e1f829ca2195c88'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:22.394375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4622e92dbf747049e1f829ca2195c88'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:22.396899 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4622e92dbf747049e1f829ca2195c88'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:22.411867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4622e92dbf747049e1f829ca2195c88'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:22.436311 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4622e92dbf747049e1f829ca2195c88'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:22.481520 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1133/2276] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:22 2022] DELETE /identity/v3/users/b4622e92dbf747049e1f829ca2195c88 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:22.486886 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9759f0-8d81-4341-8ff2-8fbe294d3bd9 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:22.487913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9759f0-8d81-4341-8ff2-8fbe294d3bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:22.489283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9759f0-8d81-4341-8ff2-8fbe294d3bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:22.576760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0a9759f0-8d81-4341-8ff2-8fbe294d3bd9 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:22.585180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0a9759f0-8d81-4341-8ff2-8fbe294d3bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:22.686508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:22.808373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1144/2277] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:22.820340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:22.822530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:23.095018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846faac-045e-4b0d-aa8b-0d783585a2aa admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:23.101261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc9d1c-e540-471a-999c-b73696b48f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:23.101961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc9d1c-e540-471a-999c-b73696b48f68 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:23.102659 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc9d1c-e540-471a-999c-b73696b48f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:23.103071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc9d1c-e540-471a-999c-b73696b48f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:23.138196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc9d1c-e540-471a-999c-b73696b48f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:23.139672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc9d1c-e540-471a-999c-b73696b48f68 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:23.154013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1134/2278] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:23.332479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a36e-b87b-4734-9d7c-9777e4b9e289 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:23.332777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a36e-b87b-4734-9d7c-9777e4b9e289 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:23.333069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a36e-b87b-4734-9d7c-9777e4b9e289 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:23.336149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1145/2279] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:16:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:16:23.356932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ced892-ecec-4705-8d6a-26aa081dc584 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:23.356932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ced892-ecec-4705-8d6a-26aa081dc584 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:23.356932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ced892-ecec-4705-8d6a-26aa081dc584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:23.424516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-89ced892-ecec-4705-8d6a-26aa081dc584 None None] MFA Rules not processed for user `aff2305249a242639e1415094bf9df33`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:23.426844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-89ced892-ecec-4705-8d6a-26aa081dc584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:23.541176 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1135/2280] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:16:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:23.560276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3a36e-b87b-4734-9d7c-9777e4b9e289 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:23.570400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-86e3a36e-b87b-4734-9d7c-9777e4b9e289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:23.593776 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ced892-ecec-4705-8d6a-26aa081dc584 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:23.595594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-89ced892-ecec-4705-8d6a-26aa081dc584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:23.607948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3a36e-b87b-4734-9d7c-9777e4b9e289 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:23.638409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ced892-ecec-4705-8d6a-26aa081dc584 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:23.646144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:23.647475 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:23.648712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:23.649303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] PATH_INFO: `/v3/users/cb99f1e73a214b4ab32102a9ab5670fb` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:23.650524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ba517-4318-452b-a0f4-e5dab303dad0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:23.651738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba517-4318-452b-a0f4-e5dab303dad0 service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:23.651911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb99f1e73a214b4ab32102a9ab5670fb)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:23.652077 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba517-4318-452b-a0f4-e5dab303dad0 service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:23.652524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba517-4318-452b-a0f4-e5dab303dad0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:23.655947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:23.674923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ba517-4318-452b-a0f4-e5dab303dad0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:23.678306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ba517-4318-452b-a0f4-e5dab303dad0 service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:23.687212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:23.704449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1146/2281] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:16:23.708387 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ba517-4318-452b-a0f4-e5dab303dad0 service placement] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:23.710598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9a1ba517-4318-452b-a0f4-e5dab303dad0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:23.728383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ba517-4318-452b-a0f4-e5dab303dad0 service placement] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:23.734402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:23.735297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:23.735585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:23.735862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] PATH_INFO: `/v3/projects/68d2a8094f8446a0a86e05c76bb6dc87` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:23.737658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d2a8094f8446a0a86e05c76bb6dc87)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:23.739668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:23.754275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb99f1e73a214b4ab32102a9ab5670fb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:23.755359 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb99f1e73a214b4ab32102a9ab5670fb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:23.756050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb99f1e73a214b4ab32102a9ab5670fb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:23.770241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb99f1e73a214b4ab32102a9ab5670fb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:23.788454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb99f1e73a214b4ab32102a9ab5670fb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:23.805984 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d2a8094f8446a0a86e05c76bb6dc87', 'request_id': 'req-cfc448af-5e03-49ec-a413-8fbae76d30c2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:23.819234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d2a8094f8446a0a86e05c76bb6dc87', 'request_id': 'req-cfc448af-5e03-49ec-a413-8fbae76d30c2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:23.832134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d2a8094f8446a0a86e05c76bb6dc87', 'request_id': 'req-cfc448af-5e03-49ec-a413-8fbae76d30c2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:23.839510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1136/2282] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:23 2022] DELETE /identity/v3/users/cb99f1e73a214b4ab32102a9ab5670fb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:23.857335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] The token cache is being invalidate because project 68d2a8094f8446a0a86e05c76bb6dc87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:23.858063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:23.858786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cfc448af-5e03-49ec-a413-8fbae76d30c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:23.860406 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1147/2283] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:23 2022] DELETE /identity/v3/projects/68d2a8094f8446a0a86e05c76bb6dc87 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:24.398574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:24.401120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:24.524093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde038bb-18c1-4274-b649-a9abd99cd751 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:24.530311 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:24.531468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:24.532094 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:24.532557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] PATH_INFO: `/v3/projects/13c95b6f91e84cd498cc57c55253a477` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:24.534551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13c95b6f91e84cd498cc57c55253a477)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:24.538045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:24.613244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13c95b6f91e84cd498cc57c55253a477', 'request_id': 'req-11431112-7577-48bb-820d-58e4a5dcf132'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:24.625910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13c95b6f91e84cd498cc57c55253a477', 'request_id': 'req-11431112-7577-48bb-820d-58e4a5dcf132'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:24.640065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13c95b6f91e84cd498cc57c55253a477', 'request_id': 'req-11431112-7577-48bb-820d-58e4a5dcf132'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:24.656429 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] The token cache is being invalidate because project 13c95b6f91e84cd498cc57c55253a477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:24.656794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:24.657558 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-11431112-7577-48bb-820d-58e4a5dcf132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:24.659769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1137/2284] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:24 2022] DELETE /identity/v3/projects/13c95b6f91e84cd498cc57c55253a477 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:24.670820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4666e-4f70-4edf-96b2-1ce1057728c7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:24.672101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4666e-4f70-4edf-96b2-1ce1057728c7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:24.672585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4666e-4f70-4edf-96b2-1ce1057728c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:24.697351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d0b4666e-4f70-4edf-96b2-1ce1057728c7 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:24.698694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d0b4666e-4f70-4edf-96b2-1ce1057728c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:24.806052 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1148/2285] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:24.813478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca377c1-2e68-43cd-986c-cb56ae76eef3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:24.813759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca377c1-2e68-43cd-986c-cb56ae76eef3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:24.814074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca377c1-2e68-43cd-986c-cb56ae76eef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:24.844365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cca377c1-2e68-43cd-986c-cb56ae76eef3 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:24.845575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cca377c1-2e68-43cd-986c-cb56ae76eef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:24.854698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1138/2286] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:24.859233 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4666e-4f70-4edf-96b2-1ce1057728c7 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:24.871185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4666e-4f70-4edf-96b2-1ce1057728c7 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:24.876608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48fa23-ce6b-48a6-9353-912ef925ab7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:24.877331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48fa23-ce6b-48a6-9353-912ef925ab7f admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:24.877621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48fa23-ce6b-48a6-9353-912ef925ab7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:24.877960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48fa23-ce6b-48a6-9353-912ef925ab7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:24.878888 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48fa23-ce6b-48a6-9353-912ef925ab7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:24.879202 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48fa23-ce6b-48a6-9353-912ef925ab7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:24.880845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48fa23-ce6b-48a6-9353-912ef925ab7f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:24.893670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1149/2287] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:16:24.898284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca377c1-2e68-43cd-986c-cb56ae76eef3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:24.909754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca377c1-2e68-43cd-986c-cb56ae76eef3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:24.914829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc4f81-19ed-4bbd-becc-5dbc2698b514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:24.915501 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc4f81-19ed-4bbd-becc-5dbc2698b514 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:24.915779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc4f81-19ed-4bbd-becc-5dbc2698b514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:24.916051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc4f81-19ed-4bbd-becc-5dbc2698b514 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:24.916750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc4f81-19ed-4bbd-becc-5dbc2698b514 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:24.918720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc4f81-19ed-4bbd-becc-5dbc2698b514 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:24.933411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:24.951288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1139/2288] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:24 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:16:24.952605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48fa23-ce6b-48a6-9353-912ef925ab7f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:24.964192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48fa23-ce6b-48a6-9353-912ef925ab7f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:24.997753 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:25.034533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b1df4-0986-4941-926b-5559229644da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:25.035515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b1df4-0986-4941-926b-5559229644da admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:25.035807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b1df4-0986-4941-926b-5559229644da admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:25.036088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b1df4-0986-4941-926b-5559229644da admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:25.036786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b1df4-0986-4941-926b-5559229644da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:25.039619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b1df4-0986-4941-926b-5559229644da admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:25.070327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1150/2289] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:24 2022] POST /identity/v3/users => generated 303 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:25.071506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc4f81-19ed-4bbd-becc-5dbc2698b514 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:25.082347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc4f81-19ed-4bbd-becc-5dbc2698b514 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:25.090749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976efe7-cffd-4d8a-9e5f-6f625841d4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:25.091437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976efe7-cffd-4d8a-9e5f-6f625841d4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:25.091746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976efe7-cffd-4d8a-9e5f-6f625841d4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:25.092041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976efe7-cffd-4d8a-9e5f-6f625841d4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:25.092783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976efe7-cffd-4d8a-9e5f-6f625841d4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:25.094531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976efe7-cffd-4d8a-9e5f-6f625841d4d5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:25.106533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1140/2290] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:25.110541 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b1df4-0986-4941-926b-5559229644da admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:25.126654 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b1df4-0986-4941-926b-5559229644da admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:25.141501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482a9ff-f967-42a7-96bf-c84282e1ce74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:25.142440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482a9ff-f967-42a7-96bf-c84282e1ce74 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:25.142908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482a9ff-f967-42a7-96bf-c84282e1ce74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:25.143338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482a9ff-f967-42a7-96bf-c84282e1ce74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:25.146428 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482a9ff-f967-42a7-96bf-c84282e1ce74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:25.148419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482a9ff-f967-42a7-96bf-c84282e1ce74 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:25.173809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1151/2291] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:25.177656 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976efe7-cffd-4d8a-9e5f-6f625841d4d5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:25.188320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976efe7-cffd-4d8a-9e5f-6f625841d4d5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:25.194332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5bc11-cc3b-433f-b063-ec453a18a09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:25.195234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5bc11-cc3b-433f-b063-ec453a18a09d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:25.195514 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5bc11-cc3b-433f-b063-ec453a18a09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:25.195788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5bc11-cc3b-433f-b063-ec453a18a09d admin admin] PATH_INFO: `/v3/projects/2d8dffe8102f4f63be31c18ea8dc9c54/users/6417ca1797ef4217925f01de7ecac0cb/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:25.212140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5bc11-cc3b-433f-b063-ec453a18a09d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d8dffe8102f4f63be31c18ea8dc9c54, user_id=6417ca1797ef4217925f01de7ecac0cb, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:25.214839 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5bc11-cc3b-433f-b063-ec453a18a09d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:25.235973 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1141/2292] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:25 2022] PUT /identity/v3/projects/2d8dffe8102f4f63be31c18ea8dc9c54/users/6417ca1797ef4217925f01de7ecac0cb/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:25.243009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb228b8-a2b4-4e79-842b-4a169d45e3d0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:25.243317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb228b8-a2b4-4e79-842b-4a169d45e3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:25.244187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb228b8-a2b4-4e79-842b-4a169d45e3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:25.264135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-3cb228b8-a2b4-4e79-842b-4a169d45e3d0 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:25.270324 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3cb228b8-a2b4-4e79-842b-4a169d45e3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:25.350421 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1152/2293] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:25.369433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5bc11-cc3b-433f-b063-ec453a18a09d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:25.371384 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4fc5bc11-cc3b-433f-b063-ec453a18a09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:25.457450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5bc11-cc3b-433f-b063-ec453a18a09d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:25.462043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee18f2-3395-424c-ac5d-1209a0169bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:25.463054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee18f2-3395-424c-ac5d-1209a0169bcf service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:25.463054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee18f2-3395-424c-ac5d-1209a0169bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:25.463790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee18f2-3395-424c-ac5d-1209a0169bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:25.478896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee18f2-3395-424c-ac5d-1209a0169bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:25.480639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee18f2-3395-424c-ac5d-1209a0169bcf service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:25.496072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1142/2294] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:26.552209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86267-f461-4ab2-b606-9182384d80b3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:26.552954 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86267-f461-4ab2-b606-9182384d80b3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:26.553245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86267-f461-4ab2-b606-9182384d80b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:26.585810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-61f86267-f461-4ab2-b606-9182384d80b3 None None] MFA Rules not processed for user `6417ca1797ef4217925f01de7ecac0cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:26.587420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-61f86267-f461-4ab2-b606-9182384d80b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:26.656615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1153/2295] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:26.661896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee18f2-3395-424c-ac5d-1209a0169bcf service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:26.672663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee18f2-3395-424c-ac5d-1209a0169bcf service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:26.678954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e2146-59e4-41ab-ac98-0cf2aba9ceef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:26.679694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e2146-59e4-41ab-ac98-0cf2aba9ceef admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:26.680040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e2146-59e4-41ab-ac98-0cf2aba9ceef admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:26.680497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e2146-59e4-41ab-ac98-0cf2aba9ceef admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:26.681208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e2146-59e4-41ab-ac98-0cf2aba9ceef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:26.683168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e2146-59e4-41ab-ac98-0cf2aba9ceef admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:26.713683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1143/2296] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:26 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:16:26.718153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f86267-f461-4ab2-b606-9182384d80b3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:26.730936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f86267-f461-4ab2-b606-9182384d80b3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:26.763440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:26.792252 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346eb443-99ea-41f1-a87d-68aed3694977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:26.793110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346eb443-99ea-41f1-a87d-68aed3694977 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:26.793336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346eb443-99ea-41f1-a87d-68aed3694977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:26.793609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346eb443-99ea-41f1-a87d-68aed3694977 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:26.794900 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346eb443-99ea-41f1-a87d-68aed3694977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:26.796962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346eb443-99ea-41f1-a87d-68aed3694977 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:26.819869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1154/2297] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:26 2022] POST /identity/v3/users => generated 309 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:26.824282 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e2146-59e4-41ab-ac98-0cf2aba9ceef admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:26.835266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e2146-59e4-41ab-ac98-0cf2aba9ceef admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:26.839915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccacb6-66da-4dcc-a471-dc79c769504d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:26.840598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccacb6-66da-4dcc-a471-dc79c769504d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:26.840881 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccacb6-66da-4dcc-a471-dc79c769504d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:26.841156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccacb6-66da-4dcc-a471-dc79c769504d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:26.841923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccacb6-66da-4dcc-a471-dc79c769504d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:26.843539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccacb6-66da-4dcc-a471-dc79c769504d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:26.854570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1144/2298] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:26.859024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346eb443-99ea-41f1-a87d-68aed3694977 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:26.869753 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346eb443-99ea-41f1-a87d-68aed3694977 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:26.875135 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a26c94-9a47-4793-9f1c-274694854a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:26.876041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a26c94-9a47-4793-9f1c-274694854a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:26.876323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a26c94-9a47-4793-9f1c-274694854a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:26.876983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a26c94-9a47-4793-9f1c-274694854a27 admin admin] PATH_INFO: `/v3/projects/f919460dd4254358bdf9737b116e4569/users/f605fb6c7c1d4e449105283d64a0b0b6/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:26.899525 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a26c94-9a47-4793-9f1c-274694854a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f919460dd4254358bdf9737b116e4569, user_id=f605fb6c7c1d4e449105283d64a0b0b6, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:26.904132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a26c94-9a47-4793-9f1c-274694854a27 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:26.923834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1155/2299] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:26 2022] PUT /identity/v3/projects/f919460dd4254358bdf9737b116e4569/users/f605fb6c7c1d4e449105283d64a0b0b6/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:16:26.928419 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccacb6-66da-4dcc-a471-dc79c769504d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:26.939498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccacb6-66da-4dcc-a471-dc79c769504d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:26.997727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6239d-1da0-46ff-9670-855cabbc7780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:26.998464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6239d-1da0-46ff-9670-855cabbc7780 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:26.998746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6239d-1da0-46ff-9670-855cabbc7780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:26.999019 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6239d-1da0-46ff-9670-855cabbc7780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:26.999751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6239d-1da0-46ff-9670-855cabbc7780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:27.001370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6239d-1da0-46ff-9670-855cabbc7780 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:27.011804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1145/2300] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:27.016177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a26c94-9a47-4793-9f1c-274694854a27 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:27.026876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a26c94-9a47-4793-9f1c-274694854a27 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:27.031662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e1d24-8bdc-471c-8f70-212ce2b522fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:27.032545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e1d24-8bdc-471c-8f70-212ce2b522fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:27.032828 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e1d24-8bdc-471c-8f70-212ce2b522fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:27.033132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e1d24-8bdc-471c-8f70-212ce2b522fd admin admin] PATH_INFO: `/v3/projects/f919460dd4254358bdf9737b116e4569/users/f605fb6c7c1d4e449105283d64a0b0b6/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:27.035262 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e1d24-8bdc-471c-8f70-212ce2b522fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f919460dd4254358bdf9737b116e4569, user_id=f605fb6c7c1d4e449105283d64a0b0b6, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:27.037926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e1d24-8bdc-471c-8f70-212ce2b522fd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:27.049706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1156/2301] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:27 2022] PUT /identity/v3/projects/f919460dd4254358bdf9737b116e4569/users/f605fb6c7c1d4e449105283d64a0b0b6/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:16:28.059458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9debbba-0dd0-4ed5-bdc6-e9dc0e747585 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:28.059949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9debbba-0dd0-4ed5-bdc6-e9dc0e747585 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:28.060160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9debbba-0dd0-4ed5-bdc6-e9dc0e747585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:28.109681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f9debbba-0dd0-4ed5-bdc6-e9dc0e747585 None None] MFA Rules not processed for user `f605fb6c7c1d4e449105283d64a0b0b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:28.111150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f9debbba-0dd0-4ed5-bdc6-e9dc0e747585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:28.153703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:28.299704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1146/2302] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:28 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:28.862384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45419b97-10b9-4cfd-9e90-15298cd2204b None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:28.862919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45419b97-10b9-4cfd-9e90-15298cd2204b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:28.863011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45419b97-10b9-4cfd-9e90-15298cd2204b None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:28.864385 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1157/2303] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:16:28.875653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85a78b-1d03-4566-9abc-de70c7d4f4b2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:28.876044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85a78b-1d03-4566-9abc-de70c7d4f4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:28.876329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85a78b-1d03-4566-9abc-de70c7d4f4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:28.901248 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cb85a78b-1d03-4566-9abc-de70c7d4f4b2 None None] MFA Rules not processed for user `aff2305249a242639e1415094bf9df33`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:28.902537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cb85a78b-1d03-4566-9abc-de70c7d4f4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:28.985037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1147/2304] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:16:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:28.999823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45419b97-10b9-4cfd-9e90-15298cd2204b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:29.002096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-45419b97-10b9-4cfd-9e90-15298cd2204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:29.028097 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45419b97-10b9-4cfd-9e90-15298cd2204b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:29.034689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd9701-d7e4-402f-b740-84f3acf5b80a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:29.035655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd9701-d7e4-402f-b740-84f3acf5b80a service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:29.036107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd9701-d7e4-402f-b740-84f3acf5b80a service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:29.036548 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd9701-d7e4-402f-b740-84f3acf5b80a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:29.046289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd9701-d7e4-402f-b740-84f3acf5b80a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:29.047954 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd9701-d7e4-402f-b740-84f3acf5b80a service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:29.060282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1158/2305] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:29.540449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85a78b-1d03-4566-9abc-de70c7d4f4b2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:29.624813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85a78b-1d03-4566-9abc-de70c7d4f4b2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:29.630763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:29.631782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:29.632069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:29.632339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] PATH_INFO: `/v3/users/6417ca1797ef4217925f01de7ecac0cb` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:29.633583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6417ca1797ef4217925f01de7ecac0cb)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:29.636044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:29.694308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6417ca1797ef4217925f01de7ecac0cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:29.694969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6417ca1797ef4217925f01de7ecac0cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:29.695575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6417ca1797ef4217925f01de7ecac0cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:29.698134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:29.705280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6417ca1797ef4217925f01de7ecac0cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:29.714385 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6417ca1797ef4217925f01de7ecac0cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:29.732553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1148/2306] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:29 2022] DELETE /identity/v3/users/6417ca1797ef4217925f01de7ecac0cb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:29.912408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd9701-d7e4-402f-b740-84f3acf5b80a service placement] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:29.925590 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd9701-d7e4-402f-b740-84f3acf5b80a service placement] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:29.999305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:30.000204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:30.000484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:30.000756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] PATH_INFO: `/v3/projects/2d8dffe8102f4f63be31c18ea8dc9c54` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:30.002117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d8dffe8102f4f63be31c18ea8dc9c54)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:30.004070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:30.058146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d8dffe8102f4f63be31c18ea8dc9c54', 'request_id': 'req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.072794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d8dffe8102f4f63be31c18ea8dc9c54', 'request_id': 'req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.081491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d8dffe8102f4f63be31c18ea8dc9c54', 'request_id': 'req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.093370 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] The token cache is being invalidate because project 2d8dffe8102f4f63be31c18ea8dc9c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:30.093793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.094546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.096304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1159/2307] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:29 2022] DELETE /identity/v3/projects/2d8dffe8102f4f63be31c18ea8dc9c54 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:30.100374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:30.101932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:30.115253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3af90-eb22-45a1-9a83-0a3fef8916d5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:30.185405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c908c9-84c2-49e4-b3b9-d5e5a2f7f366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:30.186386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c908c9-84c2-49e4-b3b9-d5e5a2f7f366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:30.186915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c908c9-84c2-49e4-b3b9-d5e5a2f7f366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:30.187207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c908c9-84c2-49e4-b3b9-d5e5a2f7f366 admin admin] PATH_INFO: `/v3/users/f605fb6c7c1d4e449105283d64a0b0b6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:30.188513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c908c9-84c2-49e4-b3b9-d5e5a2f7f366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f605fb6c7c1d4e449105283d64a0b0b6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:30.191383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c908c9-84c2-49e4-b3b9-d5e5a2f7f366 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:30.251225 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-87c908c9-84c2-49e4-b3b9-d5e5a2f7f366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f605fb6c7c1d4e449105283d64a0b0b6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.252012 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-87c908c9-84c2-49e4-b3b9-d5e5a2f7f366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f605fb6c7c1d4e449105283d64a0b0b6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.253185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-87c908c9-84c2-49e4-b3b9-d5e5a2f7f366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f605fb6c7c1d4e449105283d64a0b0b6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.261586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-87c908c9-84c2-49e4-b3b9-d5e5a2f7f366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f605fb6c7c1d4e449105283d64a0b0b6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.269814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-87c908c9-84c2-49e4-b3b9-d5e5a2f7f366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f605fb6c7c1d4e449105283d64a0b0b6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.290250 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1149/2308] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:30 2022] DELETE /identity/v3/users/f605fb6c7c1d4e449105283d64a0b0b6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:30.544272 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:30.545976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:30.555527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:30.560193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cb635-c5f1-4ef6-8377-5c13d8a8fb35 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:30.645937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:30.646820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:30.647101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:30.647378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] PATH_INFO: `/v3/projects/f919460dd4254358bdf9737b116e4569` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:30.648553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f919460dd4254358bdf9737b116e4569)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:30.650513 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:30.710058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f919460dd4254358bdf9737b116e4569', 'request_id': 'req-26dbc2d5-1928-4916-9524-082cb74312cc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.721473 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f919460dd4254358bdf9737b116e4569', 'request_id': 'req-26dbc2d5-1928-4916-9524-082cb74312cc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.731260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f919460dd4254358bdf9737b116e4569', 'request_id': 'req-26dbc2d5-1928-4916-9524-082cb74312cc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.744853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] The token cache is being invalidate because project f919460dd4254358bdf9737b116e4569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:30.745158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.746335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-26dbc2d5-1928-4916-9524-082cb74312cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:30.748960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1160/2309] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:30 2022] DELETE /identity/v3/projects/f919460dd4254358bdf9737b116e4569 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:30.755627 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25576f0-0b63-44f8-bd09-64952fdf2a27 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:30.756222 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25576f0-0b63-44f8-bd09-64952fdf2a27 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:30.756536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25576f0-0b63-44f8-bd09-64952fdf2a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:30.779635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b25576f0-0b63-44f8-bd09-64952fdf2a27 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:30.780856 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b25576f0-0b63-44f8-bd09-64952fdf2a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:30.821713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:30.841532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1150/2310] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:16:30.849628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858260f-05bd-41fe-9f88-6aa210d92ae4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:30.849960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858260f-05bd-41fe-9f88-6aa210d92ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:30.850246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858260f-05bd-41fe-9f88-6aa210d92ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:30.871201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e858260f-05bd-41fe-9f88-6aa210d92ae4 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:30.872456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e858260f-05bd-41fe-9f88-6aa210d92ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:30.882157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1161/2311] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:30.886779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25576f0-0b63-44f8-bd09-64952fdf2a27 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:30.896761 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25576f0-0b63-44f8-bd09-64952fdf2a27 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:30.902072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cfafb-56e1-4592-9481-b56f93512192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:30.902748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cfafb-56e1-4592-9481-b56f93512192 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:30.903036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cfafb-56e1-4592-9481-b56f93512192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:30.903311 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cfafb-56e1-4592-9481-b56f93512192 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:30.904049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cfafb-56e1-4592-9481-b56f93512192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:30.904700 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cfafb-56e1-4592-9481-b56f93512192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:30.906566 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cfafb-56e1-4592-9481-b56f93512192 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:30.919152 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1151/2312] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:16:30.923251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858260f-05bd-41fe-9f88-6aa210d92ae4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:30.933149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858260f-05bd-41fe-9f88-6aa210d92ae4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:30.938036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7fee6-4de8-4106-b472-0a5c82cff598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:30.938696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7fee6-4de8-4106-b472-0a5c82cff598 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:30.938999 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7fee6-4de8-4106-b472-0a5c82cff598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:30.939233 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7fee6-4de8-4106-b472-0a5c82cff598 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:30.939955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7fee6-4de8-4106-b472-0a5c82cff598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:30.941858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7fee6-4de8-4106-b472-0a5c82cff598 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:30.970706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1162/2313] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:30 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:30.975437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cfafb-56e1-4592-9481-b56f93512192 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:30.986115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cfafb-56e1-4592-9481-b56f93512192 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:31.049338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f7ac3-ac0b-489d-b6a3-6e9769db929d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:31.050120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f7ac3-ac0b-489d-b6a3-6e9769db929d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.050418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f7ac3-ac0b-489d-b6a3-6e9769db929d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.050719 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f7ac3-ac0b-489d-b6a3-6e9769db929d admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.051474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f7ac3-ac0b-489d-b6a3-6e9769db929d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:31.053492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f7ac3-ac0b-489d-b6a3-6e9769db929d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:31.080296 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1152/2314] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:30 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:31.084706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7fee6-4de8-4106-b472-0a5c82cff598 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:31.095323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7fee6-4de8-4106-b472-0a5c82cff598 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:31.100223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39537511-ca2b-4d3c-bc62-330f697a1df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:31.100881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537511-ca2b-4d3c-bc62-330f697a1df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.101163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537511-ca2b-4d3c-bc62-330f697a1df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.101435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537511-ca2b-4d3c-bc62-330f697a1df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.102197 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39537511-ca2b-4d3c-bc62-330f697a1df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:31.103805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39537511-ca2b-4d3c-bc62-330f697a1df3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:31.114429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1163/2315] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:31.118449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f7ac3-ac0b-489d-b6a3-6e9769db929d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:31.129628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f7ac3-ac0b-489d-b6a3-6e9769db929d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:31.135339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e2898-17ed-472a-b63f-e63c68266255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:31.136023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e2898-17ed-472a-b63f-e63c68266255 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.136342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e2898-17ed-472a-b63f-e63c68266255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.136617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e2898-17ed-472a-b63f-e63c68266255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.137358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e2898-17ed-472a-b63f-e63c68266255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:31.139027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e2898-17ed-472a-b63f-e63c68266255 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:31.150001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1153/2316] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:31.154494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39537511-ca2b-4d3c-bc62-330f697a1df3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:31.165461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39537511-ca2b-4d3c-bc62-330f697a1df3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:31.171293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c26ee-ef99-4d93-bc53-f71ab773f578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:31.172187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c26ee-ef99-4d93-bc53-f71ab773f578 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.172470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c26ee-ef99-4d93-bc53-f71ab773f578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.172745 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c26ee-ef99-4d93-bc53-f71ab773f578 admin admin] PATH_INFO: `/v3/projects/24950e6905294a9cb1fe6570683121d2/users/d88a5cb1eabd4fd68d7f017774111fc0/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.189395 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c26ee-ef99-4d93-bc53-f71ab773f578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24950e6905294a9cb1fe6570683121d2, user_id=d88a5cb1eabd4fd68d7f017774111fc0, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:31.192211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c26ee-ef99-4d93-bc53-f71ab773f578 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:31.208981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1164/2317] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:31 2022] PUT /identity/v3/projects/24950e6905294a9cb1fe6570683121d2/users/d88a5cb1eabd4fd68d7f017774111fc0/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:31.223354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68250730-3eb2-4485-bb4a-92f70a0dead2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.223650 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68250730-3eb2-4485-bb4a-92f70a0dead2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.223920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68250730-3eb2-4485-bb4a-92f70a0dead2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.254664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-68250730-3eb2-4485-bb4a-92f70a0dead2 None None] MFA Rules not processed for user `d88a5cb1eabd4fd68d7f017774111fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:31.255877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-68250730-3eb2-4485-bb4a-92f70a0dead2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:31.329740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1154/2318] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:31 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:31.334757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c26ee-ef99-4d93-bc53-f71ab773f578 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:31.371213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:31.405657 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c26ee-ef99-4d93-bc53-f71ab773f578 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:31.410851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affbe3d-e094-43ff-b720-14e3ed486e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:31.411599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affbe3d-e094-43ff-b720-14e3ed486e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.411851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affbe3d-e094-43ff-b720-14e3ed486e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.412127 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affbe3d-e094-43ff-b720-14e3ed486e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.412882 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affbe3d-e094-43ff-b720-14e3ed486e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:31.414866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affbe3d-e094-43ff-b720-14e3ed486e26 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:31.450791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1165/2319] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:31 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:31.451974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68250730-3eb2-4485-bb4a-92f70a0dead2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:31.465840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68250730-3eb2-4485-bb4a-92f70a0dead2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:31.525436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:31.553809 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc545f-c30b-4bca-a15f-79556ee258ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:31.555237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc545f-c30b-4bca-a15f-79556ee258ee admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.555533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc545f-c30b-4bca-a15f-79556ee258ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.555817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc545f-c30b-4bca-a15f-79556ee258ee admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.556855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc545f-c30b-4bca-a15f-79556ee258ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:31.559408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc545f-c30b-4bca-a15f-79556ee258ee admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:31.594737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1155/2320] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:31 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:16:31.599106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affbe3d-e094-43ff-b720-14e3ed486e26 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:31.613325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affbe3d-e094-43ff-b720-14e3ed486e26 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:31.619340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ec6ff-697e-43b6-ba69-9f0adfd7cc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:31.620103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ec6ff-697e-43b6-ba69-9f0adfd7cc22 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.620458 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ec6ff-697e-43b6-ba69-9f0adfd7cc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.622131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ec6ff-697e-43b6-ba69-9f0adfd7cc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.622131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ec6ff-697e-43b6-ba69-9f0adfd7cc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:31.623122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ec6ff-697e-43b6-ba69-9f0adfd7cc22 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:31.638186 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1166/2321] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:31.639934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc545f-c30b-4bca-a15f-79556ee258ee admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:31.654833 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc545f-c30b-4bca-a15f-79556ee258ee admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:31.661134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2564057-dd41-4e13-8701-b06040915236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:31.662098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2564057-dd41-4e13-8701-b06040915236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.662383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2564057-dd41-4e13-8701-b06040915236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.662657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2564057-dd41-4e13-8701-b06040915236 admin admin] PATH_INFO: `/v3/projects/a1003d77a8cc407993ff93b5dfce4120/users/2d77ff4740bd4745bdac50eaee039039/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.680111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2564057-dd41-4e13-8701-b06040915236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1003d77a8cc407993ff93b5dfce4120, user_id=2d77ff4740bd4745bdac50eaee039039, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:31.683581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2564057-dd41-4e13-8701-b06040915236 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:31.704132 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1156/2322] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:31 2022] PUT /identity/v3/projects/a1003d77a8cc407993ff93b5dfce4120/users/2d77ff4740bd4745bdac50eaee039039/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:16:31.708663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ec6ff-697e-43b6-ba69-9f0adfd7cc22 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:31.720361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ec6ff-697e-43b6-ba69-9f0adfd7cc22 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:31.781162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cafe5-6dc1-469a-b469-0112ba9f716f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:31.781845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cafe5-6dc1-469a-b469-0112ba9f716f admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.782158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cafe5-6dc1-469a-b469-0112ba9f716f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.782484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cafe5-6dc1-469a-b469-0112ba9f716f admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.783391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cafe5-6dc1-469a-b469-0112ba9f716f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:31.785183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cafe5-6dc1-469a-b469-0112ba9f716f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:31.800489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1167/2323] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:16:31.802265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2564057-dd41-4e13-8701-b06040915236 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:31.815180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2564057-dd41-4e13-8701-b06040915236 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:31.819926 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c48323-b161-4a3a-b982-4363db5ef4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:31.820863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c48323-b161-4a3a-b982-4363db5ef4d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.821150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c48323-b161-4a3a-b982-4363db5ef4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.821430 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c48323-b161-4a3a-b982-4363db5ef4d1 admin admin] PATH_INFO: `/v3/projects/a1003d77a8cc407993ff93b5dfce4120/users/2d77ff4740bd4745bdac50eaee039039/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.823469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c48323-b161-4a3a-b982-4363db5ef4d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1003d77a8cc407993ff93b5dfce4120, user_id=2d77ff4740bd4745bdac50eaee039039, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:31.826183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c48323-b161-4a3a-b982-4363db5ef4d1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:31.837057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1157/2324] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:31 2022] PUT /identity/v3/projects/a1003d77a8cc407993ff93b5dfce4120/users/2d77ff4740bd4745bdac50eaee039039/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:31.848456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208fdd1-aba4-4647-a637-66dc00bd5e02 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:31.848749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208fdd1-aba4-4647-a637-66dc00bd5e02 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:31.849034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208fdd1-aba4-4647-a637-66dc00bd5e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:31.879010 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9208fdd1-aba4-4647-a637-66dc00bd5e02 None None] MFA Rules not processed for user `2d77ff4740bd4745bdac50eaee039039`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:31.880244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9208fdd1-aba4-4647-a637-66dc00bd5e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:31.996693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1168/2325] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:31 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:32.011705 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c48323-b161-4a3a-b982-4363db5ef4d1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:32.013497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-07c48323-b161-4a3a-b982-4363db5ef4d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:32.094394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c48323-b161-4a3a-b982-4363db5ef4d1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:32.103510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f51e3-6b4e-4a5b-b107-220061b868ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:32.104252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f51e3-6b4e-4a5b-b107-220061b868ba service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:32.104739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f51e3-6b4e-4a5b-b107-220061b868ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:32.105216 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f51e3-6b4e-4a5b-b107-220061b868ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:32.122822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f51e3-6b4e-4a5b-b107-220061b868ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:32.131641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f51e3-6b4e-4a5b-b107-220061b868ba service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:32.147010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1158/2326] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:32 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:33.060494 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208fdd1-aba4-4647-a637-66dc00bd5e02 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:33.062812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9208fdd1-aba4-4647-a637-66dc00bd5e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:33.113467 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:33.158985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208fdd1-aba4-4647-a637-66dc00bd5e02 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:33.164679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:33.165586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:33.166055 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:33.166228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] PATH_INFO: `/v3/users/85894c323fa94bcbb5585a7ae85cc355` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:33.167710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] RBAC: Authorizing `identity:delete_user(user_id=85894c323fa94bcbb5585a7ae85cc355)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:33.169686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:33.254298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85894c323fa94bcbb5585a7ae85cc355'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.254982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85894c323fa94bcbb5585a7ae85cc355'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.265047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85894c323fa94bcbb5585a7ae85cc355'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.274836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85894c323fa94bcbb5585a7ae85cc355'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.275772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85894c323fa94bcbb5585a7ae85cc355'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.297813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1169/2327] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:33 2022] DELETE /identity/v3/users/85894c323fa94bcbb5585a7ae85cc355 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:33.543393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f51e3-6b4e-4a5b-b107-220061b868ba service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:33.570150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f51e3-6b4e-4a5b-b107-220061b868ba service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:33.648444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:33.650156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:33.650622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:33.651097 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] PATH_INFO: `/v3/projects/77f9faeaa8334099ba0c171b8250b78c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:33.652836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=77f9faeaa8334099ba0c171b8250b78c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:33.654970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:33.729515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:33.739248 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77f9faeaa8334099ba0c171b8250b78c', 'request_id': 'req-b9454efc-01c9-42d3-90d0-f71e8c4349ce'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.751555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77f9faeaa8334099ba0c171b8250b78c', 'request_id': 'req-b9454efc-01c9-42d3-90d0-f71e8c4349ce'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.759076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77f9faeaa8334099ba0c171b8250b78c', 'request_id': 'req-b9454efc-01c9-42d3-90d0-f71e8c4349ce'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.775044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] The token cache is being invalidate because project 77f9faeaa8334099ba0c171b8250b78c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:33.775337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.776022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.777592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1159/2328] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:33 2022] DELETE /identity/v3/projects/77f9faeaa8334099ba0c171b8250b78c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:33.781623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:33.783060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:33.800944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c1446-4c9e-40d4-be94-1411a7a2e44f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:33.910174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ff0b2-cfb0-4fbf-9eaf-4443365acdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:33.910174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff0b2-cfb0-4fbf-9eaf-4443365acdb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:33.910174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff0b2-cfb0-4fbf-9eaf-4443365acdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:33.910174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff0b2-cfb0-4fbf-9eaf-4443365acdb7 admin admin] PATH_INFO: `/v3/users/364a7e1a20364a58b1c4669cce7aacee` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:33.910632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ff0b2-cfb0-4fbf-9eaf-4443365acdb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=364a7e1a20364a58b1c4669cce7aacee)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:33.910632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ff0b2-cfb0-4fbf-9eaf-4443365acdb7 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:33.913067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:33.914617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:33.929411 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9454efc-01c9-42d3-90d0-f71e8c4349ce admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:33.935100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:33.936111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:33.936450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:33.936826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] PATH_INFO: `/v3/users/d88a5cb1eabd4fd68d7f017774111fc0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:33.938163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] RBAC: Authorizing `identity:delete_user(user_id=d88a5cb1eabd4fd68d7f017774111fc0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:33.941006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:33.968824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-025ff0b2-cfb0-4fbf-9eaf-4443365acdb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364a7e1a20364a58b1c4669cce7aacee'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.969683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-025ff0b2-cfb0-4fbf-9eaf-4443365acdb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364a7e1a20364a58b1c4669cce7aacee'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.972110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:33.980867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-025ff0b2-cfb0-4fbf-9eaf-4443365acdb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364a7e1a20364a58b1c4669cce7aacee'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.989976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-025ff0b2-cfb0-4fbf-9eaf-4443365acdb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364a7e1a20364a58b1c4669cce7aacee'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.990795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-025ff0b2-cfb0-4fbf-9eaf-4443365acdb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364a7e1a20364a58b1c4669cce7aacee'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.998822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd88a5cb1eabd4fd68d7f017774111fc0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:33.999603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd88a5cb1eabd4fd68d7f017774111fc0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.000302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd88a5cb1eabd4fd68d7f017774111fc0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.012333 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd88a5cb1eabd4fd68d7f017774111fc0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.015016 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1170/2329] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:33 2022] DELETE /identity/v3/users/364a7e1a20364a58b1c4669cce7aacee => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:34.029544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd88a5cb1eabd4fd68d7f017774111fc0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.056587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1160/2330] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:33 2022] DELETE /identity/v3/users/d88a5cb1eabd4fd68d7f017774111fc0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:34.061170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0f692-5127-4a3f-9d15-2f7594f880ec None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.061676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0f692-5127-4a3f-9d15-2f7594f880ec None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.061771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0f692-5127-4a3f-9d15-2f7594f880ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.081942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1bb0f692-5127-4a3f-9d15-2f7594f880ec None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:34.083683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1bb0f692-5127-4a3f-9d15-2f7594f880ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:34.163440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1171/2331] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:16:34.176681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:34.182925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:34.266721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b2160-2a40-499e-bf16-c1e7ca1b17be admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:34.267362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0f692-5127-4a3f-9d15-2f7594f880ec None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:34.269297 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1bb0f692-5127-4a3f-9d15-2f7594f880ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:34.278658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90294770-571c-4564-8515-39f78fa156b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:34.280336 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90294770-571c-4564-8515-39f78fa156b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.280853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90294770-571c-4564-8515-39f78fa156b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.282149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90294770-571c-4564-8515-39f78fa156b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.290247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0f692-5127-4a3f-9d15-2f7594f880ec None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:34.298655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:34.300607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.300888 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.301159 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] PATH_INFO: `/v3/projects/e49b405054e44158a62083dcd2ece0d7` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.302450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e49b405054e44158a62083dcd2ece0d7)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:34.305300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:34.306567 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90294770-571c-4564-8515-39f78fa156b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:34.308177 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90294770-571c-4564-8515-39f78fa156b2 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:34.333801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1161/2332] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:34.385634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e49b405054e44158a62083dcd2ece0d7', 'request_id': 'req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.396677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e49b405054e44158a62083dcd2ece0d7', 'request_id': 'req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.406302 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e49b405054e44158a62083dcd2ece0d7', 'request_id': 'req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.425789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] The token cache is being invalidate because project e49b405054e44158a62083dcd2ece0d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:34.426210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.427209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5f51af32-d47c-4dde-9e5c-c59f4d6131c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.429741 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1172/2333] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:34 2022] DELETE /identity/v3/projects/e49b405054e44158a62083dcd2ece0d7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:16:34.436168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1737d18-0618-432b-be3f-2d07f2fbdd7d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.436492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1737d18-0618-432b-be3f-2d07f2fbdd7d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.436776 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1737d18-0618-432b-be3f-2d07f2fbdd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.457871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d1737d18-0618-432b-be3f-2d07f2fbdd7d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:34.459288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d1737d18-0618-432b-be3f-2d07f2fbdd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:34.546249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:34.577430 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1162/2334] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:34.589257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5f4e8-7e9e-4eb5-9bac-563e398867dc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.589749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5f4e8-7e9e-4eb5-9bac-563e398867dc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.590765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5f4e8-7e9e-4eb5-9bac-563e398867dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.637609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-92c5f4e8-7e9e-4eb5-9bac-563e398867dc None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:34.639895 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-92c5f4e8-7e9e-4eb5-9bac-563e398867dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:34.652264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1173/2335] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:34.656999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1737d18-0618-432b-be3f-2d07f2fbdd7d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:34.666914 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5f4e8-7e9e-4eb5-9bac-563e398867dc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:34.668679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-92c5f4e8-7e9e-4eb5-9bac-563e398867dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:34.669431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1737d18-0618-432b-be3f-2d07f2fbdd7d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:34.675015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9363b12-470a-4051-8461-6b33bbea430b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:34.675783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9363b12-470a-4051-8461-6b33bbea430b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.676232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9363b12-470a-4051-8461-6b33bbea430b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.677044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9363b12-470a-4051-8461-6b33bbea430b admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.678191 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9363b12-470a-4051-8461-6b33bbea430b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:34.679521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9363b12-470a-4051-8461-6b33bbea430b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:34.681966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9363b12-470a-4051-8461-6b33bbea430b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:34.683094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5f4e8-7e9e-4eb5-9bac-563e398867dc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:34.694087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:34.695228 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.695569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.695908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] PATH_INFO: `/v3/projects/24950e6905294a9cb1fe6570683121d2` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.696839 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1163/2336] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:16:34.697338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24950e6905294a9cb1fe6570683121d2)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:34.699290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:34.700747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9363b12-470a-4051-8461-6b33bbea430b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:34.711401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9363b12-470a-4051-8461-6b33bbea430b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:34.716197 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22095bd-2606-4b78-b538-b6cd906fcf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:34.716934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22095bd-2606-4b78-b538-b6cd906fcf3c admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.717276 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22095bd-2606-4b78-b538-b6cd906fcf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.717613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22095bd-2606-4b78-b538-b6cd906fcf3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.718435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22095bd-2606-4b78-b538-b6cd906fcf3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:34.720439 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22095bd-2606-4b78-b538-b6cd906fcf3c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:34.746628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:34.748186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1164/2337] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:34 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:16:34.752601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22095bd-2606-4b78-b538-b6cd906fcf3c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:34.761335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24950e6905294a9cb1fe6570683121d2', 'request_id': 'req-02895a86-95bf-4e36-af17-17eefe4b1d02'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.770270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24950e6905294a9cb1fe6570683121d2', 'request_id': 'req-02895a86-95bf-4e36-af17-17eefe4b1d02'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.787355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24950e6905294a9cb1fe6570683121d2', 'request_id': 'req-02895a86-95bf-4e36-af17-17eefe4b1d02'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.800694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] The token cache is being invalidate because project 24950e6905294a9cb1fe6570683121d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:34.800988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.801823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.804496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1174/2338] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:34 2022] DELETE /identity/v3/projects/24950e6905294a9cb1fe6570683121d2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:34.808383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:34.809792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:34.820119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22095bd-2606-4b78-b538-b6cd906fcf3c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:34.825255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277da0e-a3e1-4d3d-8feb-bdbfaa9cd23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:34.826057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277da0e-a3e1-4d3d-8feb-bdbfaa9cd23c admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.826353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277da0e-a3e1-4d3d-8feb-bdbfaa9cd23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.826657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277da0e-a3e1-4d3d-8feb-bdbfaa9cd23c admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.827406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277da0e-a3e1-4d3d-8feb-bdbfaa9cd23c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:34.829363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277da0e-a3e1-4d3d-8feb-bdbfaa9cd23c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:34.851455 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1165/2339] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:34 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:34.855412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277da0e-a3e1-4d3d-8feb-bdbfaa9cd23c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:34.857018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c277da0e-a3e1-4d3d-8feb-bdbfaa9cd23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:34.868990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277da0e-a3e1-4d3d-8feb-bdbfaa9cd23c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:34.873688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cebdb-e325-4bae-9847-094cdb677cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:34.874381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cebdb-e325-4bae-9847-094cdb677cde admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.874666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cebdb-e325-4bae-9847-094cdb677cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.874944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cebdb-e325-4bae-9847-094cdb677cde admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.875674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cebdb-e325-4bae-9847-094cdb677cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:34.877545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cebdb-e325-4bae-9847-094cdb677cde admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:34.880231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02895a86-95bf-4e36-af17-17eefe4b1d02 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:34.885249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:34.886154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.886487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.886742 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] PATH_INFO: `/v3/users/2d77ff4740bd4745bdac50eaee039039` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.887981 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d77ff4740bd4745bdac50eaee039039)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:34.888158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1166/2340] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:16:34.891715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:34.893742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cebdb-e325-4bae-9847-094cdb677cde admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:34.902938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cebdb-e325-4bae-9847-094cdb677cde admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:34.907554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5be6a2-1f05-44cd-983c-2e34584d4923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:34.908365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5be6a2-1f05-44cd-983c-2e34584d4923 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.908655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5be6a2-1f05-44cd-983c-2e34584d4923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.908933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5be6a2-1f05-44cd-983c-2e34584d4923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.909654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5be6a2-1f05-44cd-983c-2e34584d4923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:34.911304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5be6a2-1f05-44cd-983c-2e34584d4923 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:34.923960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1167/2341] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:34.928725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5be6a2-1f05-44cd-983c-2e34584d4923 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:34.943550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5be6a2-1f05-44cd-983c-2e34584d4923 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:34.950031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6e739-3860-4b53-9a57-f3cfda67a1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:34.952638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e739-3860-4b53-9a57-f3cfda67a1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.952638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e739-3860-4b53-9a57-f3cfda67a1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.952638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e739-3860-4b53-9a57-f3cfda67a1b3 admin admin] PATH_INFO: `/v3/projects/fec91e0c883943448c9d0e8d4d5e09ae/users/c0ebb8aa3b7146e4a34a565969154f9a/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:34.964068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d77ff4740bd4745bdac50eaee039039'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.964810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d77ff4740bd4745bdac50eaee039039'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.971051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6e739-3860-4b53-9a57-f3cfda67a1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec91e0c883943448c9d0e8d4d5e09ae, user_id=c0ebb8aa3b7146e4a34a565969154f9a, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:34.973672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6e739-3860-4b53-9a57-f3cfda67a1b3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:34.977617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d77ff4740bd4745bdac50eaee039039'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.988327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d77ff4740bd4745bdac50eaee039039'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.988988 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1168/2342] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:34 2022] PUT /identity/v3/projects/fec91e0c883943448c9d0e8d4d5e09ae/users/c0ebb8aa3b7146e4a34a565969154f9a/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:34.989298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d77ff4740bd4745bdac50eaee039039'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:34.994161 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2d580-be64-4866-91e3-d22800d7b304 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:34.994778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2d580-be64-4866-91e3-d22800d7b304 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:34.995065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2d580-be64-4866-91e3-d22800d7b304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:35.015066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1175/2343] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:34 2022] DELETE /identity/v3/users/2d77ff4740bd4745bdac50eaee039039 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:16:35.024208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d6d2d580-be64-4866-91e3-d22800d7b304 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:35.025488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d6d2d580-be64-4866-91e3-d22800d7b304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:35.091404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:35.120649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1169/2344] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:16:35.132166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:35.136333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:35.190334 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:35.204528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb8c71-cffa-45af-a1ac-5fb851410578 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:35.208572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fd03f-adde-4214-b083-ad4b757dbfd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:35.209236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fd03f-adde-4214-b083-ad4b757dbfd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:35.209523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fd03f-adde-4214-b083-ad4b757dbfd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:35.209804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fd03f-adde-4214-b083-ad4b757dbfd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:35.220505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fd03f-adde-4214-b083-ad4b757dbfd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:35.222110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fd03f-adde-4214-b083-ad4b757dbfd1 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:35.233440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1176/2345] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:35.532603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2d580-be64-4866-91e3-d22800d7b304 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:35.534194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d6d2d580-be64-4866-91e3-d22800d7b304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:35.548612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2d580-be64-4866-91e3-d22800d7b304 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:35.554389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:35.555349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:35.555664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:35.555950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] PATH_INFO: `/v3/projects/a1003d77a8cc407993ff93b5dfce4120` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:35.557392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1003d77a8cc407993ff93b5dfce4120)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:35.559456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:35.648782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1003d77a8cc407993ff93b5dfce4120', 'request_id': 'req-1e703fa2-1d97-432c-88a0-15990bb55449'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:35.652581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1003d77a8cc407993ff93b5dfce4120', 'request_id': 'req-1e703fa2-1d97-432c-88a0-15990bb55449'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:35.660841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1003d77a8cc407993ff93b5dfce4120', 'request_id': 'req-1e703fa2-1d97-432c-88a0-15990bb55449'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:35.676324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] The token cache is being invalidate because project a1003d77a8cc407993ff93b5dfce4120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:35.676938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:35.677683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1e703fa2-1d97-432c-88a0-15990bb55449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:35.679605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1170/2346] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:35 2022] DELETE /identity/v3/projects/a1003d77a8cc407993ff93b5dfce4120 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:35.686049 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a562b53-7ff0-4df6-9499-0828486938d0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:35.686619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a562b53-7ff0-4df6-9499-0828486938d0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:35.687226 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a562b53-7ff0-4df6-9499-0828486938d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:35.728945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5a562b53-7ff0-4df6-9499-0828486938d0 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:35.730384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5a562b53-7ff0-4df6-9499-0828486938d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:35.821967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1177/2347] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:35.833614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd8d6-70ae-418f-8935-6f16328d9703 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:35.833614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd8d6-70ae-418f-8935-6f16328d9703 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:35.833974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd8d6-70ae-418f-8935-6f16328d9703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:35.865228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3bfdd8d6-70ae-418f-8935-6f16328d9703 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:35.866428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3bfdd8d6-70ae-418f-8935-6f16328d9703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:35.875338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1171/2348] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:35.879785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a562b53-7ff0-4df6-9499-0828486938d0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:35.890037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a562b53-7ff0-4df6-9499-0828486938d0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:35.895122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3432b9-abea-4e02-b091-6e5fe9f7be00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:35.895801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3432b9-abea-4e02-b091-6e5fe9f7be00 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:35.896084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3432b9-abea-4e02-b091-6e5fe9f7be00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:35.896352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3432b9-abea-4e02-b091-6e5fe9f7be00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:35.897087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3432b9-abea-4e02-b091-6e5fe9f7be00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:35.897441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3432b9-abea-4e02-b091-6e5fe9f7be00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:35.899268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3432b9-abea-4e02-b091-6e5fe9f7be00 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:35.918019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1178/2349] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:16:35.919139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdd8d6-70ae-418f-8935-6f16328d9703 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:35.929388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdd8d6-70ae-418f-8935-6f16328d9703 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:35.934249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d636f3-57e4-4424-9fbf-c1be489874d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:35.934917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d636f3-57e4-4424-9fbf-c1be489874d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:35.935202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d636f3-57e4-4424-9fbf-c1be489874d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:35.935471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d636f3-57e4-4424-9fbf-c1be489874d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:35.936181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d636f3-57e4-4424-9fbf-c1be489874d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:35.938145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d636f3-57e4-4424-9fbf-c1be489874d1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:35.971328 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1172/2350] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:35 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:35.975927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3432b9-abea-4e02-b091-6e5fe9f7be00 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:35.988047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3432b9-abea-4e02-b091-6e5fe9f7be00 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:36.077110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576ee88-de2b-49eb-876a-f1020a5fb7ab None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:36.077398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576ee88-de2b-49eb-876a-f1020a5fb7ab None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:36.077676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576ee88-de2b-49eb-876a-f1020a5fb7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:36.094659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eb007-e91d-47ba-9958-251ccd38a05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:36.095389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb007-e91d-47ba-9958-251ccd38a05e admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:36.095876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb007-e91d-47ba-9958-251ccd38a05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:36.096212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb007-e91d-47ba-9958-251ccd38a05e admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:36.096946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eb007-e91d-47ba-9958-251ccd38a05e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:36.099034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eb007-e91d-47ba-9958-251ccd38a05e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:36.139521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1179/2351] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:35 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:16:36.144149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7576ee88-de2b-49eb-876a-f1020a5fb7ab None None] MFA Rules not processed for user `c0ebb8aa3b7146e4a34a565969154f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:36.145305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7576ee88-de2b-49eb-876a-f1020a5fb7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:36.152145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eb007-e91d-47ba-9958-251ccd38a05e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:36.180325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eb007-e91d-47ba-9958-251ccd38a05e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:36.201533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e393648-079e-4d46-ac00-ec615d5e6049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:36.203095 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393648-079e-4d46-ac00-ec615d5e6049 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:36.203379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393648-079e-4d46-ac00-ec615d5e6049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:36.203637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393648-079e-4d46-ac00-ec615d5e6049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:36.204403 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e393648-079e-4d46-ac00-ec615d5e6049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:36.206118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e393648-079e-4d46-ac00-ec615d5e6049 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:36.237775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1180/2352] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:36.246034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e393648-079e-4d46-ac00-ec615d5e6049 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:36.259492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e393648-079e-4d46-ac00-ec615d5e6049 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:36.269726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabea8d3-a7f7-4018-b73e-ef3f61df1cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:36.270858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabea8d3-a7f7-4018-b73e-ef3f61df1cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:36.270858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabea8d3-a7f7-4018-b73e-ef3f61df1cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:36.271062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabea8d3-a7f7-4018-b73e-ef3f61df1cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:36.271723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabea8d3-a7f7-4018-b73e-ef3f61df1cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:36.278231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabea8d3-a7f7-4018-b73e-ef3f61df1cf3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:36.295172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1173/2353] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:36.303775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1181/2354] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:36.309459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576ee88-de2b-49eb-876a-f1020a5fb7ab None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:36.325190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576ee88-de2b-49eb-876a-f1020a5fb7ab None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:36.330098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b81ea-0cd2-4a7c-88c4-31c9b7e6b0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:36.331071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b81ea-0cd2-4a7c-88c4-31c9b7e6b0e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:36.331502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b81ea-0cd2-4a7c-88c4-31c9b7e6b0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:36.331785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b81ea-0cd2-4a7c-88c4-31c9b7e6b0e2 admin admin] PATH_INFO: `/v3/projects/58cac7b282604f519bf31545c2c68ca6/users/7ffaed6ca4094b438a30c6c82b8efc8e/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:36.348308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b81ea-0cd2-4a7c-88c4-31c9b7e6b0e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58cac7b282604f519bf31545c2c68ca6, user_id=7ffaed6ca4094b438a30c6c82b8efc8e, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:36.351413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b81ea-0cd2-4a7c-88c4-31c9b7e6b0e2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:36.367368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1174/2355] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:36 2022] PUT /identity/v3/projects/58cac7b282604f519bf31545c2c68ca6/users/7ffaed6ca4094b438a30c6c82b8efc8e/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:36.377718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c77da0-ca49-462f-87e4-a87533961025 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:36.378082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c77da0-ca49-462f-87e4-a87533961025 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:36.378212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c77da0-ca49-462f-87e4-a87533961025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:36.412441 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-34c77da0-ca49-462f-87e4-a87533961025 None None] MFA Rules not processed for user `7ffaed6ca4094b438a30c6c82b8efc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:36.413988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-34c77da0-ca49-462f-87e4-a87533961025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:36.451235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:36.496704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1182/2356] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:36.501331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b81ea-0cd2-4a7c-88c4-31c9b7e6b0e2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:36.516104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:36.617499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b81ea-0cd2-4a7c-88c4-31c9b7e6b0e2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:36.624987 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca962d16-c978-4d41-9dea-899dcd7bd472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:36.626188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca962d16-c978-4d41-9dea-899dcd7bd472 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:36.626655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca962d16-c978-4d41-9dea-899dcd7bd472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:36.627337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca962d16-c978-4d41-9dea-899dcd7bd472 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:36.627544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c77da0-ca49-462f-87e4-a87533961025 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:36.628354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca962d16-c978-4d41-9dea-899dcd7bd472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:36.629595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-34c77da0-ca49-462f-87e4-a87533961025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:36.631442 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca962d16-c978-4d41-9dea-899dcd7bd472 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:36.694104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1175/2357] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:36 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:16:36.695663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca962d16-c978-4d41-9dea-899dcd7bd472 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:36.752967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c77da0-ca49-462f-87e4-a87533961025 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:36.785383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca962d16-c978-4d41-9dea-899dcd7bd472 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:36.790460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5717aff-1c2e-4942-87c3-0a46866e54df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:36.791287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5717aff-1c2e-4942-87c3-0a46866e54df admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:36.791615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5717aff-1c2e-4942-87c3-0a46866e54df admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:36.791844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5717aff-1c2e-4942-87c3-0a46866e54df admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:36.793085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5717aff-1c2e-4942-87c3-0a46866e54df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:36.795471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5717aff-1c2e-4942-87c3-0a46866e54df admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:36.819555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1176/2358] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:36 2022] POST /identity/v3/users => generated 297 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:16:36.823652 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5717aff-1c2e-4942-87c3-0a46866e54df admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:36.825447 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973db4e-c40b-4d6f-9cfd-d713950bf81c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:36.826140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973db4e-c40b-4d6f-9cfd-d713950bf81c service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:36.826422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973db4e-c40b-4d6f-9cfd-d713950bf81c service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:36.826780 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973db4e-c40b-4d6f-9cfd-d713950bf81c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:36.836444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5717aff-1c2e-4942-87c3-0a46866e54df admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:36.842767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de861c-f2ee-45b1-bff3-c80a10075a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:36.843721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de861c-f2ee-45b1-bff3-c80a10075a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:36.843933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de861c-f2ee-45b1-bff3-c80a10075a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:36.844329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de861c-f2ee-45b1-bff3-c80a10075a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:36.846386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de861c-f2ee-45b1-bff3-c80a10075a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:36.848232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de861c-f2ee-45b1-bff3-c80a10075a45 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:36.861304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1177/2359] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:16:36.866749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de861c-f2ee-45b1-bff3-c80a10075a45 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:36.879021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de861c-f2ee-45b1-bff3-c80a10075a45 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:36.883868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fa79f-61ed-4f5e-bd1b-04595a471471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:36.885093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fa79f-61ed-4f5e-bd1b-04595a471471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:36.885375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fa79f-61ed-4f5e-bd1b-04595a471471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:36.885653 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fa79f-61ed-4f5e-bd1b-04595a471471 admin admin] PATH_INFO: `/v3/projects/c6819774fcc44d338ae34796ac00c90c/users/2fa92a89eb1a4be0a14626f4944f665d/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:36.892295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973db4e-c40b-4d6f-9cfd-d713950bf81c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:36.894122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973db4e-c40b-4d6f-9cfd-d713950bf81c service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:36.904584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1183/2360] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:36.907287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819fa79f-61ed-4f5e-bd1b-04595a471471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6819774fcc44d338ae34796ac00c90c, user_id=2fa92a89eb1a4be0a14626f4944f665d, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:36.910065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819fa79f-61ed-4f5e-bd1b-04595a471471 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:36.926867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1178/2361] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:36 2022] PUT /identity/v3/projects/c6819774fcc44d338ae34796ac00c90c/users/2fa92a89eb1a4be0a14626f4944f665d/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:36.931344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973db4e-c40b-4d6f-9cfd-d713950bf81c service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:36.943512 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973db4e-c40b-4d6f-9cfd-d713950bf81c service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:36.991964 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:37.011694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b25c7-314c-42be-8b33-e863b00a615d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:37.012372 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b25c7-314c-42be-8b33-e863b00a615d admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:37.012656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b25c7-314c-42be-8b33-e863b00a615d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:37.012930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b25c7-314c-42be-8b33-e863b00a615d admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:37.013676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b25c7-314c-42be-8b33-e863b00a615d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:37.015343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b25c7-314c-42be-8b33-e863b00a615d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:37.028695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1184/2362] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:37.029955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fa79f-61ed-4f5e-bd1b-04595a471471 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:37.041094 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fa79f-61ed-4f5e-bd1b-04595a471471 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:37.046836 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5a15b-877f-4bab-a659-61b3b74cb397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:37.048680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5a15b-877f-4bab-a659-61b3b74cb397 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:37.049026 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5a15b-877f-4bab-a659-61b3b74cb397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:37.049310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5a15b-877f-4bab-a659-61b3b74cb397 admin admin] PATH_INFO: `/v3/projects/c6819774fcc44d338ae34796ac00c90c/users/2fa92a89eb1a4be0a14626f4944f665d/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:37.052606 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5a15b-877f-4bab-a659-61b3b74cb397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6819774fcc44d338ae34796ac00c90c, user_id=2fa92a89eb1a4be0a14626f4944f665d, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:37.057137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5a15b-877f-4bab-a659-61b3b74cb397 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:37.072971 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1179/2363] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:37 2022] PUT /identity/v3/projects/c6819774fcc44d338ae34796ac00c90c/users/2fa92a89eb1a4be0a14626f4944f665d/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:37.085388 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62f009-fbb1-43f6-bbed-086d772e74c8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:37.086434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62f009-fbb1-43f6-bbed-086d772e74c8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:37.086749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62f009-fbb1-43f6-bbed-086d772e74c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:37.133390 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2a62f009-fbb1-43f6-bbed-086d772e74c8 None None] MFA Rules not processed for user `2fa92a89eb1a4be0a14626f4944f665d`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:37.135109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2a62f009-fbb1-43f6-bbed-086d772e74c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:37.229408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1185/2364] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:37.246747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5a15b-877f-4bab-a659-61b3b74cb397 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:37.311320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:37.334546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5a15b-877f-4bab-a659-61b3b74cb397 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:37.338602 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e2e56-09d1-4c95-9190-f741fe061d76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:37.339268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2e56-09d1-4c95-9190-f741fe061d76 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:37.339550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2e56-09d1-4c95-9190-f741fe061d76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:37.339830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e2e56-09d1-4c95-9190-f741fe061d76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:37.350196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e2e56-09d1-4c95-9190-f741fe061d76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:37.351684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e2e56-09d1-4c95-9190-f741fe061d76 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:37.361305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1180/2365] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:37.455738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62f009-fbb1-43f6-bbed-086d772e74c8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:37.467223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62f009-fbb1-43f6-bbed-086d772e74c8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:37.471562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ea6b2-ad4e-4f05-bbec-881c7c473481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:37.472216 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ea6b2-ad4e-4f05-bbec-881c7c473481 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:37.472499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ea6b2-ad4e-4f05-bbec-881c7c473481 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:37.472776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ea6b2-ad4e-4f05-bbec-881c7c473481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:37.536400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ea6b2-ad4e-4f05-bbec-881c7c473481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:37.538172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ea6b2-ad4e-4f05-bbec-881c7c473481 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:37.550293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1186/2366] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:38.411403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c61f8-9d82-4052-99e5-f9ef94b54606 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:38.411946 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c61f8-9d82-4052-99e5-f9ef94b54606 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:38.411946 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c61f8-9d82-4052-99e5-f9ef94b54606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:38.432118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d19c61f8-9d82-4052-99e5-f9ef94b54606 None None] MFA Rules not processed for user `c0ebb8aa3b7146e4a34a565969154f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:38.433350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d19c61f8-9d82-4052-99e5-f9ef94b54606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:38.490137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1181/2367] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:40.349927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ea6b2-ad4e-4f05-bbec-881c7c473481 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:40.495075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ea6b2-ad4e-4f05-bbec-881c7c473481 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:40.500014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:40.500928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:40.501237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:40.501510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] PATH_INFO: `/v3/users/7ffaed6ca4094b438a30c6c82b8efc8e` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:40.502864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ffaed6ca4094b438a30c6c82b8efc8e)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:40.504863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:40.507757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:40.559878 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ffaed6ca4094b438a30c6c82b8efc8e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:40.560557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ffaed6ca4094b438a30c6c82b8efc8e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:40.569410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ffaed6ca4094b438a30c6c82b8efc8e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:40.576514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ffaed6ca4094b438a30c6c82b8efc8e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:40.577169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ffaed6ca4094b438a30c6c82b8efc8e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:40.598375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1187/2368] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:40 2022] DELETE /identity/v3/users/7ffaed6ca4094b438a30c6c82b8efc8e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:40.603104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed0b5-e86d-4b9c-bf9e-26bd4f5417a0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:40.603601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed0b5-e86d-4b9c-bf9e-26bd4f5417a0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:40.604004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed0b5-e86d-4b9c-bf9e-26bd4f5417a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:40.626788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d8fed0b5-e86d-4b9c-bf9e-26bd4f5417a0 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:40.628127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d8fed0b5-e86d-4b9c-bf9e-26bd4f5417a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:40.690196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1182/2369] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:40.699851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:40.701307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:40.767891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c05b87-3f01-4423-9c33-88246c59795b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:40.771654 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533006bf-cc8e-4aa3-b50b-ccf5e62466ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:40.772312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533006bf-cc8e-4aa3-b50b-ccf5e62466ec service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:40.772594 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533006bf-cc8e-4aa3-b50b-ccf5e62466ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:40.772872 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533006bf-cc8e-4aa3-b50b-ccf5e62466ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:40.782149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533006bf-cc8e-4aa3-b50b-ccf5e62466ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:40.783731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533006bf-cc8e-4aa3-b50b-ccf5e62466ec service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:40.794434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1188/2370] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:41.146117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed0b5-e86d-4b9c-bf9e-26bd4f5417a0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:41.148056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d8fed0b5-e86d-4b9c-bf9e-26bd4f5417a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:41.171832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed0b5-e86d-4b9c-bf9e-26bd4f5417a0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:41.186057 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:41.187806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:41.188165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:41.188517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] PATH_INFO: `/v3/projects/58cac7b282604f519bf31545c2c68ca6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:41.189847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58cac7b282604f519bf31545c2c68ca6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:41.192474 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:41.255010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58cac7b282604f519bf31545c2c68ca6', 'request_id': 'req-26515752-9adc-4555-ad83-f238fe666ec1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:41.264071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58cac7b282604f519bf31545c2c68ca6', 'request_id': 'req-26515752-9adc-4555-ad83-f238fe666ec1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:41.272116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58cac7b282604f519bf31545c2c68ca6', 'request_id': 'req-26515752-9adc-4555-ad83-f238fe666ec1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:41.290265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] The token cache is being invalidate because project 58cac7b282604f519bf31545c2c68ca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:41.290652 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:41.291424 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26515752-9adc-4555-ad83-f238fe666ec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:41.293274 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1183/2371] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:41 2022] DELETE /identity/v3/projects/58cac7b282604f519bf31545c2c68ca6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:41.297061 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533006bf-cc8e-4aa3-b50b-ccf5e62466ec service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:41.307952 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533006bf-cc8e-4aa3-b50b-ccf5e62466ec service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:41.377842 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:41.379430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:41.379592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:41.379912 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] PATH_INFO: `/v3/users/2fa92a89eb1a4be0a14626f4944f665d` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:41.382127 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fa92a89eb1a4be0a14626f4944f665d)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:41.384561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:41.419484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:41.456765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fa92a89eb1a4be0a14626f4944f665d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:41.457552 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fa92a89eb1a4be0a14626f4944f665d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:41.470615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fa92a89eb1a4be0a14626f4944f665d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:41.484020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fa92a89eb1a4be0a14626f4944f665d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:41.485948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fa92a89eb1a4be0a14626f4944f665d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:41.510279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1189/2372] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:41 2022] DELETE /identity/v3/users/2fa92a89eb1a4be0a14626f4944f665d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:41.539987 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09eb4-5ae2-4817-8455-d78402ff7e64 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:41.540435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09eb4-5ae2-4817-8455-d78402ff7e64 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:41.540694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09eb4-5ae2-4817-8455-d78402ff7e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:41.564506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-23d09eb4-5ae2-4817-8455-d78402ff7e64 None None] MFA Rules not processed for user `34edc297b377425b92683fc6530e8135`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:41.566220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-23d09eb4-5ae2-4817-8455-d78402ff7e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:41.571232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:41.682141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1184/2373] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:42.288056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:42.290655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:42.396598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21509e-2247-4214-8fff-28d4343148c2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:42.401260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:42.402176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:42.402458 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:42.402730 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] PATH_INFO: `/v3/projects/c6819774fcc44d338ae34796ac00c90c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:42.403929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6819774fcc44d338ae34796ac00c90c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:42.405867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:42.463585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6819774fcc44d338ae34796ac00c90c', 'request_id': 'req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:42.473836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6819774fcc44d338ae34796ac00c90c', 'request_id': 'req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:42.486092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6819774fcc44d338ae34796ac00c90c', 'request_id': 'req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:42.511471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] The token cache is being invalidate because project c6819774fcc44d338ae34796ac00c90c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:42.511471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:42.515826 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b68d5c58-bfa1-4187-9a62-f1adfb1708f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:42.523435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1190/2374] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:42 2022] DELETE /identity/v3/projects/c6819774fcc44d338ae34796ac00c90c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:42.526032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f748dd-42d6-4e62-8398-f77133ae2a12 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:42.526192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f748dd-42d6-4e62-8398-f77133ae2a12 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:42.526405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f748dd-42d6-4e62-8398-f77133ae2a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:42.575922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-27f748dd-42d6-4e62-8398-f77133ae2a12 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:42.581665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-27f748dd-42d6-4e62-8398-f77133ae2a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:42.770621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1185/2375] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:42.784428 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaa637-d2f8-42b4-867a-c6fc11bf4f6e None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:42.784723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaa637-d2f8-42b4-867a-c6fc11bf4f6e None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:42.785057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaa637-d2f8-42b4-867a-c6fc11bf4f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:42.816301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7dfaa637-d2f8-42b4-867a-c6fc11bf4f6e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:42.817609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7dfaa637-d2f8-42b4-867a-c6fc11bf4f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:42.826669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1191/2376] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:42.831646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f748dd-42d6-4e62-8398-f77133ae2a12 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:42.843253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f748dd-42d6-4e62-8398-f77133ae2a12 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:42.849627 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73685216-b85d-4a7d-9e05-b9ad8899851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:42.850399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73685216-b85d-4a7d-9e05-b9ad8899851f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:42.850630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73685216-b85d-4a7d-9e05-b9ad8899851f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:42.850923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73685216-b85d-4a7d-9e05-b9ad8899851f admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:42.851644 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73685216-b85d-4a7d-9e05-b9ad8899851f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:42.851961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73685216-b85d-4a7d-9e05-b9ad8899851f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:42.853649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73685216-b85d-4a7d-9e05-b9ad8899851f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:42.876891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaa637-d2f8-42b4-867a-c6fc11bf4f6e None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:42.878993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1186/2377] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:16:42.896278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaa637-d2f8-42b4-867a-c6fc11bf4f6e None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:42.902680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d07da-46e9-4728-809e-adba2616594c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:42.903378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d07da-46e9-4728-809e-adba2616594c admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:42.903666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d07da-46e9-4728-809e-adba2616594c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:42.903947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d07da-46e9-4728-809e-adba2616594c admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:42.904670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d07da-46e9-4728-809e-adba2616594c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:42.906840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d07da-46e9-4728-809e-adba2616594c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:42.946784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1192/2378] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:42 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:42.951582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73685216-b85d-4a7d-9e05-b9ad8899851f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:42.965488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73685216-b85d-4a7d-9e05-b9ad8899851f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:43.043857 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b878c1-ec8c-429c-96fc-da79101f150e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:43.044550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b878c1-ec8c-429c-96fc-da79101f150e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:43.044834 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b878c1-ec8c-429c-96fc-da79101f150e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:43.045110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b878c1-ec8c-429c-96fc-da79101f150e admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:43.045852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b878c1-ec8c-429c-96fc-da79101f150e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:43.048129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b878c1-ec8c-429c-96fc-da79101f150e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:43.072627 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1187/2379] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:42 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:43.076849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d07da-46e9-4728-809e-adba2616594c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:43.083927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:43.087932 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d07da-46e9-4728-809e-adba2616594c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:43.094218 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f6756-4c87-476d-8ef8-3d09f949d49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:43.095211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f6756-4c87-476d-8ef8-3d09f949d49d admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:43.095570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f6756-4c87-476d-8ef8-3d09f949d49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:43.096165 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f6756-4c87-476d-8ef8-3d09f949d49d admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:43.096983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f6756-4c87-476d-8ef8-3d09f949d49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:43.099262 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f6756-4c87-476d-8ef8-3d09f949d49d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:43.114027 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1193/2380] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:43.117076 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b878c1-ec8c-429c-96fc-da79101f150e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:43.128061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b878c1-ec8c-429c-96fc-da79101f150e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:43.133355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ab07d-76a2-4a8e-ab21-ca6fb9bc31b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:43.134014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ab07d-76a2-4a8e-ab21-ca6fb9bc31b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:43.134401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ab07d-76a2-4a8e-ab21-ca6fb9bc31b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:43.134647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ab07d-76a2-4a8e-ab21-ca6fb9bc31b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:43.136073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ab07d-76a2-4a8e-ab21-ca6fb9bc31b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:43.137841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ab07d-76a2-4a8e-ab21-ca6fb9bc31b5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:43.149702 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1188/2381] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:16:43.155284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f6756-4c87-476d-8ef8-3d09f949d49d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:43.165934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f6756-4c87-476d-8ef8-3d09f949d49d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:43.171229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c4674-86c2-4941-afa6-7fd31c6762aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:43.172178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c4674-86c2-4941-afa6-7fd31c6762aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:43.172749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c4674-86c2-4941-afa6-7fd31c6762aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:43.172976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c4674-86c2-4941-afa6-7fd31c6762aa admin admin] PATH_INFO: `/v3/projects/10018f4bb936410c9ab7317e5fde50a1/users/0899586b6cd244b98999c511bd69bc74/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:43.208074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c4674-86c2-4941-afa6-7fd31c6762aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=10018f4bb936410c9ab7317e5fde50a1, user_id=0899586b6cd244b98999c511bd69bc74, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:43.211160 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c4674-86c2-4941-afa6-7fd31c6762aa admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:43.226470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1194/2382] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:43 2022] PUT /identity/v3/projects/10018f4bb936410c9ab7317e5fde50a1/users/0899586b6cd244b98999c511bd69bc74/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:43.237341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719325a-ab7d-4476-9bbb-9c64d6ca5bfe None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:43.237661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719325a-ab7d-4476-9bbb-9c64d6ca5bfe None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:43.238148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719325a-ab7d-4476-9bbb-9c64d6ca5bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:43.267584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0719325a-ab7d-4476-9bbb-9c64d6ca5bfe None None] MFA Rules not processed for user `0899586b6cd244b98999c511bd69bc74`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:43.269206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0719325a-ab7d-4476-9bbb-9c64d6ca5bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:43.278407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:43.341201 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1189/2383] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:16:43.353189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c4674-86c2-4941-afa6-7fd31c6762aa admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:43.354949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ae4c4674-86c2-4941-afa6-7fd31c6762aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:43.416948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c4674-86c2-4941-afa6-7fd31c6762aa admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:43.421218 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216907d1-d435-4093-ba0b-a311678550a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:43.421965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216907d1-d435-4093-ba0b-a311678550a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:43.422252 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216907d1-d435-4093-ba0b-a311678550a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:43.422500 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216907d1-d435-4093-ba0b-a311678550a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:43.431338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216907d1-d435-4093-ba0b-a311678550a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:43.432908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216907d1-d435-4093-ba0b-a311678550a7 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:43.444985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1195/2384] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:43.464522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719325a-ab7d-4476-9bbb-9c64d6ca5bfe None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:43.537641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719325a-ab7d-4476-9bbb-9c64d6ca5bfe None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:43.542643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:43.543553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:43.543837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:43.544128 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] PATH_INFO: `/v3/users/0899586b6cd244b98999c511bd69bc74` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:43.545552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0899586b6cd244b98999c511bd69bc74)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:43.549431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:43.608335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0899586b6cd244b98999c511bd69bc74'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:43.609074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0899586b6cd244b98999c511bd69bc74'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:43.609827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0899586b6cd244b98999c511bd69bc74'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:43.621380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0899586b6cd244b98999c511bd69bc74'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:43.629456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0899586b6cd244b98999c511bd69bc74'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:43.648691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1190/2385] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:43 2022] DELETE /identity/v3/users/0899586b6cd244b98999c511bd69bc74 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:43.653247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9940de-0871-4b09-af07-6aaa06ce8684 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:43.653627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9940de-0871-4b09-af07-6aaa06ce8684 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:43.654015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9940de-0871-4b09-af07-6aaa06ce8684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:43.672256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4d9940de-0871-4b09-af07-6aaa06ce8684 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:43.673533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4d9940de-0871-4b09-af07-6aaa06ce8684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:43.763769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1196/2386] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:16:43.766953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:43.768909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:43.805039 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:43.837173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff19332-5a6a-41db-9e56-96f23410611b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:43.840979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fc03a-19e9-4d7c-8853-c92ad58ab949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:43.841843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fc03a-19e9-4d7c-8853-c92ad58ab949 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:43.842157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fc03a-19e9-4d7c-8853-c92ad58ab949 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:43.842484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fc03a-19e9-4d7c-8853-c92ad58ab949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:43.860013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fc03a-19e9-4d7c-8853-c92ad58ab949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:43.861562 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fc03a-19e9-4d7c-8853-c92ad58ab949 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:43.872194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1191/2387] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:44.594553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9940de-0871-4b09-af07-6aaa06ce8684 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:44.596435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4d9940de-0871-4b09-af07-6aaa06ce8684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:44.613175 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9940de-0871-4b09-af07-6aaa06ce8684 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:44.618618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:44.619974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:44.624247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:44.624729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] PATH_INFO: `/v3/projects/10018f4bb936410c9ab7317e5fde50a1` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:44.626346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] RBAC: Authorizing `identity:delete_project(project_id=10018f4bb936410c9ab7317e5fde50a1)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:44.628607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:44.682057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10018f4bb936410c9ab7317e5fde50a1', 'request_id': 'req-45309fbd-1299-4160-a9ff-230e121ef33e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:44.691422 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10018f4bb936410c9ab7317e5fde50a1', 'request_id': 'req-45309fbd-1299-4160-a9ff-230e121ef33e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:44.698610 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10018f4bb936410c9ab7317e5fde50a1', 'request_id': 'req-45309fbd-1299-4160-a9ff-230e121ef33e'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:44.703652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:44.712290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] The token cache is being invalidate because project 10018f4bb936410c9ab7317e5fde50a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:44.713025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:44.714460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-45309fbd-1299-4160-a9ff-230e121ef33e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:44.716340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1197/2388] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:44 2022] DELETE /identity/v3/projects/10018f4bb936410c9ab7317e5fde50a1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:44.723175 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34bdcc-5b70-42fc-8333-42e9118c55ce None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:44.723566 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34bdcc-5b70-42fc-8333-42e9118c55ce None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:44.723913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34bdcc-5b70-42fc-8333-42e9118c55ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:44.754717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3e34bdcc-5b70-42fc-8333-42e9118c55ce None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:44.756297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3e34bdcc-5b70-42fc-8333-42e9118c55ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:44.852993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1192/2389] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:44.861866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62af52-657b-4f0d-a797-45eeb75960bc None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:44.862410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62af52-657b-4f0d-a797-45eeb75960bc None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:44.862522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62af52-657b-4f0d-a797-45eeb75960bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:44.886941 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7d62af52-657b-4f0d-a797-45eeb75960bc None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:44.888639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7d62af52-657b-4f0d-a797-45eeb75960bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:44.898132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1198/2390] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:16:44.905197 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34bdcc-5b70-42fc-8333-42e9118c55ce None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:44.917795 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34bdcc-5b70-42fc-8333-42e9118c55ce None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:44.926012 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c7558-4b39-49c5-aa64-f6b35c872f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:44.926533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c7558-4b39-49c5-aa64-f6b35c872f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:44.926820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c7558-4b39-49c5-aa64-f6b35c872f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:44.927095 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c7558-4b39-49c5-aa64-f6b35c872f81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:44.928060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c7558-4b39-49c5-aa64-f6b35c872f81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:16:44.928179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c7558-4b39-49c5-aa64-f6b35c872f81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:44.930128 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c7558-4b39-49c5-aa64-f6b35c872f81 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:44.947965 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1193/2391] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:16:44.953910 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62af52-657b-4f0d-a797-45eeb75960bc None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:44.965742 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62af52-657b-4f0d-a797-45eeb75960bc None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:44.970808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c05f73-0b93-4b66-9d0e-6ca8f11237c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:44.971703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c05f73-0b93-4b66-9d0e-6ca8f11237c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:44.972019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c05f73-0b93-4b66-9d0e-6ca8f11237c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:44.972305 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c05f73-0b93-4b66-9d0e-6ca8f11237c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:44.973080 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c05f73-0b93-4b66-9d0e-6ca8f11237c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:44.975055 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c05f73-0b93-4b66-9d0e-6ca8f11237c0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:45.013931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1199/2392] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:16:44 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:16:45.018909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c7558-4b39-49c5-aa64-f6b35c872f81 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:45.044826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c7558-4b39-49c5-aa64-f6b35c872f81 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:45.120317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c429617-9a31-425a-a141-d28ba9836bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:45.120317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c429617-9a31-425a-a141-d28ba9836bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:45.120317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c429617-9a31-425a-a141-d28ba9836bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:45.121500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c429617-9a31-425a-a141-d28ba9836bc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:45.122287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c429617-9a31-425a-a141-d28ba9836bc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:45.125042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c429617-9a31-425a-a141-d28ba9836bc7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:45.183254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1194/2393] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:16:45 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:16:45.187444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c05f73-0b93-4b66-9d0e-6ca8f11237c0 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:45.199959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c05f73-0b93-4b66-9d0e-6ca8f11237c0 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:45.206454 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95eaf6-c888-4863-985b-7b67fafe9e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:45.207294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95eaf6-c888-4863-985b-7b67fafe9e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:45.207705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95eaf6-c888-4863-985b-7b67fafe9e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:45.208106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95eaf6-c888-4863-985b-7b67fafe9e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:45.209019 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95eaf6-c888-4863-985b-7b67fafe9e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:45.210817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95eaf6-c888-4863-985b-7b67fafe9e17 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:45.225394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1200/2394] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:16:45.229402 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c429617-9a31-425a-a141-d28ba9836bc7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:45.247179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c429617-9a31-425a-a141-d28ba9836bc7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:45.256021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40244f-f233-42d0-aa18-a33a2e154214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:45.257229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40244f-f233-42d0-aa18-a33a2e154214 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:45.257747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40244f-f233-42d0-aa18-a33a2e154214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:45.258648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40244f-f233-42d0-aa18-a33a2e154214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:45.259524 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40244f-f233-42d0-aa18-a33a2e154214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:45.261753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40244f-f233-42d0-aa18-a33a2e154214 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:45.287169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1195/2395] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:16:45.299131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95eaf6-c888-4863-985b-7b67fafe9e17 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:45.312326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95eaf6-c888-4863-985b-7b67fafe9e17 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:45.319877 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926062d-68f3-424e-86c3-4ac102ad5df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:45.321485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926062d-68f3-424e-86c3-4ac102ad5df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:45.322070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926062d-68f3-424e-86c3-4ac102ad5df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:45.322881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d926062d-68f3-424e-86c3-4ac102ad5df9 admin admin] PATH_INFO: `/v3/projects/10fc941c0bca498e9d6cfeed12f0c6ef/users/a2435e1faabf48b0922de38e93215a48/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:45.356600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926062d-68f3-424e-86c3-4ac102ad5df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10fc941c0bca498e9d6cfeed12f0c6ef, user_id=a2435e1faabf48b0922de38e93215a48, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:45.363321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d926062d-68f3-424e-86c3-4ac102ad5df9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:45.382849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1201/2396] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:16:45 2022] PUT /identity/v3/projects/10fc941c0bca498e9d6cfeed12f0c6ef/users/a2435e1faabf48b0922de38e93215a48/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:16:45.394332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652bfd8-5b42-4ff9-9e35-7a8df08eb281 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:45.394659 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652bfd8-5b42-4ff9-9e35-7a8df08eb281 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:45.394896 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652bfd8-5b42-4ff9-9e35-7a8df08eb281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:45.428207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b652bfd8-5b42-4ff9-9e35-7a8df08eb281 None None] MFA Rules not processed for user `a2435e1faabf48b0922de38e93215a48`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:45.429860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b652bfd8-5b42-4ff9-9e35-7a8df08eb281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:45.473674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:45.573759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1196/2397] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:45.582453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926062d-68f3-424e-86c3-4ac102ad5df9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:45.589011 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d926062d-68f3-424e-86c3-4ac102ad5df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:45.721687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d926062d-68f3-424e-86c3-4ac102ad5df9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:45.737531 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc31486-d19f-49d5-96d7-579805bb60d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:45.738232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc31486-d19f-49d5-96d7-579805bb60d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:45.738524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc31486-d19f-49d5-96d7-579805bb60d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:45.738804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc31486-d19f-49d5-96d7-579805bb60d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:45.750375 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc31486-d19f-49d5-96d7-579805bb60d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:45.752174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc31486-d19f-49d5-96d7-579805bb60d1 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:45.763970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1202/2398] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:47.271193 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed54b47-d2df-4eb2-9887-a4d318b428f6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:47.272391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed54b47-d2df-4eb2-9887-a4d318b428f6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:47.272706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed54b47-d2df-4eb2-9887-a4d318b428f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:47.306890 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1ed54b47-d2df-4eb2-9887-a4d318b428f6 None None] MFA Rules not processed for user `a2435e1faabf48b0922de38e93215a48`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:47.308584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1ed54b47-d2df-4eb2-9887-a4d318b428f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:47.336070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1197/2399] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:55.758283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:57.300116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=18 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:16:59.088442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc31486-d19f-49d5-96d7-579805bb60d1 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:59.090273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0fc31486-d19f-49d5-96d7-579805bb60d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:59.162330 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc31486-d19f-49d5-96d7-579805bb60d1 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:59.168230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:59.169123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:59.169436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:59.169739 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] PATH_INFO: `/v3/users/f58b8386817145f4a8d5d963444db6a2` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:59.171132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58b8386817145f4a8d5d963444db6a2)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:59.173145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:59.234048 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58b8386817145f4a8d5d963444db6a2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:59.234927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58b8386817145f4a8d5d963444db6a2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:59.246357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58b8386817145f4a8d5d963444db6a2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:59.253840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58b8386817145f4a8d5d963444db6a2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:59.254805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58b8386817145f4a8d5d963444db6a2'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:59.277088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1203/2400] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:16:59 2022] DELETE /identity/v3/users/f58b8386817145f4a8d5d963444db6a2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:59.281683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c010f82-b66c-45bf-8864-59ceabbef088 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:59.282268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c010f82-b66c-45bf-8864-59ceabbef088 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:59.282268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c010f82-b66c-45bf-8864-59ceabbef088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:59.301725 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0c010f82-b66c-45bf-8864-59ceabbef088 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:59.302968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0c010f82-b66c-45bf-8864-59ceabbef088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:59.367507 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1198/2401] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:16:59.377809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:59.379988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:59.440134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9b270-b3ad-4621-b393-e74b2e2d2703 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:59.444362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c10a8-aef8-418b-9773-273f043dd03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:59.445028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c10a8-aef8-418b-9773-273f043dd03d service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:59.445444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c10a8-aef8-418b-9773-273f043dd03d service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:59.445736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c10a8-aef8-418b-9773-273f043dd03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:59.456091 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c10a8-aef8-418b-9773-273f043dd03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:59.457823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c10a8-aef8-418b-9773-273f043dd03d service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:59.468213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1204/2402] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:16:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:16:59.778728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c010f82-b66c-45bf-8864-59ceabbef088 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:16:59.780222 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0c010f82-b66c-45bf-8864-59ceabbef088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:59.795418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c010f82-b66c-45bf-8864-59ceabbef088 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:16:59.802125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:16:59.803434 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:59.803565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:59.803908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] PATH_INFO: `/v3/projects/8ae3c1c65e6f4254958b2ea1b031a584` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:59.805798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae3c1c65e6f4254958b2ea1b031a584)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:16:59.808168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:16:59.867718 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae3c1c65e6f4254958b2ea1b031a584', 'request_id': 'req-d99a7e8e-6a83-4f48-95da-ce21ddf18195'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:59.878320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae3c1c65e6f4254958b2ea1b031a584', 'request_id': 'req-d99a7e8e-6a83-4f48-95da-ce21ddf18195'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:59.886269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae3c1c65e6f4254958b2ea1b031a584', 'request_id': 'req-d99a7e8e-6a83-4f48-95da-ce21ddf18195'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:59.897703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] The token cache is being invalidate because project 8ae3c1c65e6f4254958b2ea1b031a584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:16:59.898105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:59.898895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d99a7e8e-6a83-4f48-95da-ce21ddf18195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:16:59.908132 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1199/2403] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:16:59 2022] DELETE /identity/v3/projects/8ae3c1c65e6f4254958b2ea1b031a584 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:16:59.913003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed793c-5870-4c50-bd03-092e8a978ca1 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:16:59.913359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed793c-5870-4c50-bd03-092e8a978ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:16:59.913701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed793c-5870-4c50-bd03-092e8a978ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:16:59.934783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-44ed793c-5870-4c50-bd03-092e8a978ca1 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:16:59.936175 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-44ed793c-5870-4c50-bd03-092e8a978ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:16:59.956032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:00.008972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1205/2404] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:00.016736 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94c1f3-3c62-4c99-9952-199ac90be608 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:00.017031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94c1f3-3c62-4c99-9952-199ac90be608 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:00.017313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94c1f3-3c62-4c99-9952-199ac90be608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:00.037387 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-af94c1f3-3c62-4c99-9952-199ac90be608 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:00.038969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-af94c1f3-3c62-4c99-9952-199ac90be608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:00.058808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1200/2405] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:00.063111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed793c-5870-4c50-bd03-092e8a978ca1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:00.075853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed793c-5870-4c50-bd03-092e8a978ca1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:00.081042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3e0a6-3c78-40c9-9782-1b7bf23b7f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:00.081809 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3e0a6-3c78-40c9-9782-1b7bf23b7f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:00.082182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3e0a6-3c78-40c9-9782-1b7bf23b7f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:00.082449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3e0a6-3c78-40c9-9782-1b7bf23b7f0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:00.083527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3e0a6-3c78-40c9-9782-1b7bf23b7f0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:17:00.083885 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3e0a6-3c78-40c9-9782-1b7bf23b7f0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:00.086075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3e0a6-3c78-40c9-9782-1b7bf23b7f0a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:00.103248 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1206/2406] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:17:00.103921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94c1f3-3c62-4c99-9952-199ac90be608 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:00.114539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94c1f3-3c62-4c99-9952-199ac90be608 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:00.121792 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999bc1-47f3-46c3-a39e-8e35eade87c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:00.122569 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999bc1-47f3-46c3-a39e-8e35eade87c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:00.122856 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999bc1-47f3-46c3-a39e-8e35eade87c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:00.123162 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999bc1-47f3-46c3-a39e-8e35eade87c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:00.123865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e999bc1-47f3-46c3-a39e-8e35eade87c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:00.126574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e999bc1-47f3-46c3-a39e-8e35eade87c7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:00.177069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1201/2407] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:17:00 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:17:00.178366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3e0a6-3c78-40c9-9782-1b7bf23b7f0a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:00.189002 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3e0a6-3c78-40c9-9782-1b7bf23b7f0a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:00.263237 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a4b64-d31b-48d8-82a4-d7b9a7f551a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:00.264012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a4b64-d31b-48d8-82a4-d7b9a7f551a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:00.264369 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a4b64-d31b-48d8-82a4-d7b9a7f551a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:00.264771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a4b64-d31b-48d8-82a4-d7b9a7f551a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:00.265553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a4b64-d31b-48d8-82a4-d7b9a7f551a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:00.267673 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a4b64-d31b-48d8-82a4-d7b9a7f551a5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:00.293687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1207/2408] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:17:00 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:17:00.298007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999bc1-47f3-46c3-a39e-8e35eade87c7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:00.320537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999bc1-47f3-46c3-a39e-8e35eade87c7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:00.326532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff02398-2ee7-4db5-bf25-67c0ef41c006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:00.327303 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff02398-2ee7-4db5-bf25-67c0ef41c006 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:00.327669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff02398-2ee7-4db5-bf25-67c0ef41c006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:00.328123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff02398-2ee7-4db5-bf25-67c0ef41c006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:00.329014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff02398-2ee7-4db5-bf25-67c0ef41c006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:00.331001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff02398-2ee7-4db5-bf25-67c0ef41c006 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:00.346419 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1202/2409] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:17:00.350834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a4b64-d31b-48d8-82a4-d7b9a7f551a5 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:00.362929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a4b64-d31b-48d8-82a4-d7b9a7f551a5 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:00.369041 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f352bda-5960-458d-9d59-09cf7e8531e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:00.369864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352bda-5960-458d-9d59-09cf7e8531e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:00.370284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352bda-5960-458d-9d59-09cf7e8531e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:00.370675 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352bda-5960-458d-9d59-09cf7e8531e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:00.371565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f352bda-5960-458d-9d59-09cf7e8531e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:00.373262 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f352bda-5960-458d-9d59-09cf7e8531e8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:00.384317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1208/2410] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:17:00.388864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff02398-2ee7-4db5-bf25-67c0ef41c006 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:00.402691 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff02398-2ee7-4db5-bf25-67c0ef41c006 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:00.413841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c70ad-2dfa-491a-b4f6-3a9b181a0306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:00.415242 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c70ad-2dfa-491a-b4f6-3a9b181a0306 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:00.415714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c70ad-2dfa-491a-b4f6-3a9b181a0306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:00.416087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c70ad-2dfa-491a-b4f6-3a9b181a0306 admin admin] PATH_INFO: `/v3/projects/d3150e0802bb49fcaf5ebe0776de4cfc/users/22788abc0dbd4d7a8e1fd998e2a8f21b/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:00.436802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c70ad-2dfa-491a-b4f6-3a9b181a0306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3150e0802bb49fcaf5ebe0776de4cfc, user_id=22788abc0dbd4d7a8e1fd998e2a8f21b, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:00.439920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c70ad-2dfa-491a-b4f6-3a9b181a0306 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:00.461674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1203/2411] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:17:00 2022] PUT /identity/v3/projects/d3150e0802bb49fcaf5ebe0776de4cfc/users/22788abc0dbd4d7a8e1fd998e2a8f21b/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:17:00.473470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dad479-db8a-4b2b-8211-d50ef646c837 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:00.473950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dad479-db8a-4b2b-8211-d50ef646c837 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:00.474282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dad479-db8a-4b2b-8211-d50ef646c837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:00.510652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-67dad479-db8a-4b2b-8211-d50ef646c837 None None] MFA Rules not processed for user `22788abc0dbd4d7a8e1fd998e2a8f21b`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:00.511936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-67dad479-db8a-4b2b-8211-d50ef646c837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:00.600823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1209/2412] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:17:00.609646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c70ad-2dfa-491a-b4f6-3a9b181a0306 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:00.611804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1c2c70ad-2dfa-491a-b4f6-3a9b181a0306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:00.644166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:00.698394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c70ad-2dfa-491a-b4f6-3a9b181a0306 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:00.705209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412bf7e-6cf8-434e-ab26-00bce8226327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:00.706007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412bf7e-6cf8-434e-ab26-00bce8226327 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:00.706297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412bf7e-6cf8-434e-ab26-00bce8226327 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:00.707392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412bf7e-6cf8-434e-ab26-00bce8226327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:00.720030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412bf7e-6cf8-434e-ab26-00bce8226327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:00.721506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412bf7e-6cf8-434e-ab26-00bce8226327 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:00.741234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1204/2413] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:01.482223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c582e7-0f6b-40d2-9adb-eee9bf6076fd None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:01.482760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c582e7-0f6b-40d2-9adb-eee9bf6076fd None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:01.482760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c582e7-0f6b-40d2-9adb-eee9bf6076fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:01.501768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f1c582e7-0f6b-40d2-9adb-eee9bf6076fd None None] MFA Rules not processed for user `1e916bf18efe421895c8d7b28fcd55d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:01.503028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f1c582e7-0f6b-40d2-9adb-eee9bf6076fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:01.520813 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:01.612353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1210/2414] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:07.136955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412bf7e-6cf8-434e-ab26-00bce8226327 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:07.139599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b412bf7e-6cf8-434e-ab26-00bce8226327 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:07.213396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412bf7e-6cf8-434e-ab26-00bce8226327 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:07.217463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912ab04-deb5-480c-8d3b-f2f608498eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:07.218151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912ab04-deb5-480c-8d3b-f2f608498eec service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:07.218436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912ab04-deb5-480c-8d3b-f2f608498eec service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:07.218707 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912ab04-deb5-480c-8d3b-f2f608498eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:07.220525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0912ab04-deb5-480c-8d3b-f2f608498eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:07.294471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912ab04-deb5-480c-8d3b-f2f608498eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:07.295953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912ab04-deb5-480c-8d3b-f2f608498eec service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:07.306783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1205/2415] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:11.580939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=17 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:13.963374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c1e21-adac-4698-84fb-61721a868f63 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:13.964257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c1e21-adac-4698-84fb-61721a868f63 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:13.964257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c1e21-adac-4698-84fb-61721a868f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:13.995655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-eb2c1e21-adac-4698-84fb-61721a868f63 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:13.996923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-eb2c1e21-adac-4698-84fb-61721a868f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:14.099338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1211/2416] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:17:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:14.166107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912ab04-deb5-480c-8d3b-f2f608498eec service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:14.167612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0912ab04-deb5-480c-8d3b-f2f608498eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:14.242098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:14.280620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912ab04-deb5-480c-8d3b-f2f608498eec service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:14.285772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bca057-4442-41fd-9b58-1ec7f8ae0017 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:14.286532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca057-4442-41fd-9b58-1ec7f8ae0017 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:14.286820 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca057-4442-41fd-9b58-1ec7f8ae0017 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:14.287100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bca057-4442-41fd-9b58-1ec7f8ae0017 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:14.295958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bca057-4442-41fd-9b58-1ec7f8ae0017 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:14.297555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bca057-4442-41fd-9b58-1ec7f8ae0017 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:14.307090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1206/2417] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:17:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:17:14.701070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c1e21-adac-4698-84fb-61721a868f63 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:14.803256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c1e21-adac-4698-84fb-61721a868f63 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:14.808839 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:14.809798 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:14.810286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:14.810638 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] PATH_INFO: `/v3/users/22788abc0dbd4d7a8e1fd998e2a8f21b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:14.812138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22788abc0dbd4d7a8e1fd998e2a8f21b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:14.814681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:14.874545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22788abc0dbd4d7a8e1fd998e2a8f21b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:14.875333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22788abc0dbd4d7a8e1fd998e2a8f21b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:14.886199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22788abc0dbd4d7a8e1fd998e2a8f21b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:14.893734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:14.896870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22788abc0dbd4d7a8e1fd998e2a8f21b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:14.898208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22788abc0dbd4d7a8e1fd998e2a8f21b'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:14.923668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1212/2418] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:17:14 2022] DELETE /identity/v3/users/22788abc0dbd4d7a8e1fd998e2a8f21b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:17:14.928889 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2fd23-516b-4174-a129-b2a927b017fe None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:14.929348 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2fd23-516b-4174-a129-b2a927b017fe None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:14.929699 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2fd23-516b-4174-a129-b2a927b017fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:14.980729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-03b2fd23-516b-4174-a129-b2a927b017fe None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:14.981999 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-03b2fd23-516b-4174-a129-b2a927b017fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:15.203601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1207/2419] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:15.214112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:15.215900 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:15.380883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19516497-953f-4ba8-b431-2520ffa99557 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:15.391395 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da48ac-34e0-4bb9-bb81-f39f3d951e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:15.393057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da48ac-34e0-4bb9-bb81-f39f3d951e92 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:15.393439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da48ac-34e0-4bb9-bb81-f39f3d951e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:15.395409 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da48ac-34e0-4bb9-bb81-f39f3d951e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:15.418203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da48ac-34e0-4bb9-bb81-f39f3d951e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:15.444651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da48ac-34e0-4bb9-bb81-f39f3d951e92 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:15.450614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1213/2420] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:16.080504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2fd23-516b-4174-a129-b2a927b017fe None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:16.084613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-03b2fd23-516b-4174-a129-b2a927b017fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:16.144041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2fd23-516b-4174-a129-b2a927b017fe None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:16.164977 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:16.166498 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:16.166952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:16.167786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] PATH_INFO: `/v3/projects/d3150e0802bb49fcaf5ebe0776de4cfc` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:16.172593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3150e0802bb49fcaf5ebe0776de4cfc)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:16.175542 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:16.298892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3150e0802bb49fcaf5ebe0776de4cfc', 'request_id': 'req-70d503b5-a6a6-4e19-a291-a662542f4e4a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:16.320472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3150e0802bb49fcaf5ebe0776de4cfc', 'request_id': 'req-70d503b5-a6a6-4e19-a291-a662542f4e4a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:16.338003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3150e0802bb49fcaf5ebe0776de4cfc', 'request_id': 'req-70d503b5-a6a6-4e19-a291-a662542f4e4a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:16.363931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] The token cache is being invalidate because project d3150e0802bb49fcaf5ebe0776de4cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:17:16.364515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:16.365400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-70d503b5-a6a6-4e19-a291-a662542f4e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:16.379438 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1208/2421] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:17:16 2022] DELETE /identity/v3/projects/d3150e0802bb49fcaf5ebe0776de4cfc => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:17:16.386007 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a0d7c-4f10-4895-8928-501c89b72f5b None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:16.386892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a0d7c-4f10-4895-8928-501c89b72f5b None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:16.387331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a0d7c-4f10-4895-8928-501c89b72f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:16.433591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-094a0d7c-4f10-4895-8928-501c89b72f5b None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:16.434957 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-094a0d7c-4f10-4895-8928-501c89b72f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:16.521926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1214/2422] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:16.527101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683740a7-23ae-4a3e-94be-ee7008ea941d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:16.527637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683740a7-23ae-4a3e-94be-ee7008ea941d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:16.527924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683740a7-23ae-4a3e-94be-ee7008ea941d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:16.548584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-683740a7-23ae-4a3e-94be-ee7008ea941d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:16.550035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-683740a7-23ae-4a3e-94be-ee7008ea941d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:16.559961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1209/2423] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:16.564173 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a0d7c-4f10-4895-8928-501c89b72f5b None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:16.574393 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a0d7c-4f10-4895-8928-501c89b72f5b None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:16.579892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecaefd6-8b17-4335-b142-3cb905594b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:16.581033 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecaefd6-8b17-4335-b142-3cb905594b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:16.581529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecaefd6-8b17-4335-b142-3cb905594b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:16.582100 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecaefd6-8b17-4335-b142-3cb905594b81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:16.583070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecaefd6-8b17-4335-b142-3cb905594b81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:17:16.583586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecaefd6-8b17-4335-b142-3cb905594b81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:16.585425 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecaefd6-8b17-4335-b142-3cb905594b81 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:16.599318 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1215/2424] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:17:16.603682 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683740a7-23ae-4a3e-94be-ee7008ea941d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:16.615735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683740a7-23ae-4a3e-94be-ee7008ea941d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:16.620733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df33ec0-e0c6-4845-91af-021d04d639af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:16.621786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df33ec0-e0c6-4845-91af-021d04d639af admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:16.622297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df33ec0-e0c6-4845-91af-021d04d639af admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:16.622736 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df33ec0-e0c6-4845-91af-021d04d639af admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:16.623595 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df33ec0-e0c6-4845-91af-021d04d639af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:16.625677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df33ec0-e0c6-4845-91af-021d04d639af admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:16.657239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:16.666111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecaefd6-8b17-4335-b142-3cb905594b81 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:16.671224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1210/2425] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:17:16 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:17:16.679760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecaefd6-8b17-4335-b142-3cb905594b81 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:16.722286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:16.761449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7a6a0-d92c-4fdb-a659-f0668fb1d24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:16.762509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7a6a0-d92c-4fdb-a659-f0668fb1d24a admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:16.762882 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7a6a0-d92c-4fdb-a659-f0668fb1d24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:16.763232 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7a6a0-d92c-4fdb-a659-f0668fb1d24a admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:16.764268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7a6a0-d92c-4fdb-a659-f0668fb1d24a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:16.766399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7a6a0-d92c-4fdb-a659-f0668fb1d24a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:16.805161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1216/2426] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:17:16 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:17:16.806866 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df33ec0-e0c6-4845-91af-021d04d639af admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:16.828151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df33ec0-e0c6-4845-91af-021d04d639af admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:16.835045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba6bf6-9b44-4dc6-8f13-01104c5854c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:16.835966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba6bf6-9b44-4dc6-8f13-01104c5854c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:16.836397 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba6bf6-9b44-4dc6-8f13-01104c5854c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:16.836786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba6bf6-9b44-4dc6-8f13-01104c5854c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:16.837677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba6bf6-9b44-4dc6-8f13-01104c5854c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:16.839491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba6bf6-9b44-4dc6-8f13-01104c5854c6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:16.851902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1211/2427] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:17:16.856154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7a6a0-d92c-4fdb-a659-f0668fb1d24a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:16.872264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7a6a0-d92c-4fdb-a659-f0668fb1d24a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:16.877958 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea49fe9-5ffd-48b0-bbb9-1a5bb315a282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:16.878649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea49fe9-5ffd-48b0-bbb9-1a5bb315a282 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:16.879044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea49fe9-5ffd-48b0-bbb9-1a5bb315a282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:16.879328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea49fe9-5ffd-48b0-bbb9-1a5bb315a282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:16.880112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea49fe9-5ffd-48b0-bbb9-1a5bb315a282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:16.881739 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea49fe9-5ffd-48b0-bbb9-1a5bb315a282 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:16.896980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1217/2428] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:17:16.898364 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba6bf6-9b44-4dc6-8f13-01104c5854c6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:16.909757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba6bf6-9b44-4dc6-8f13-01104c5854c6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:16.914762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afaee3-bbdc-4990-8735-fa157cf8e55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:16.915779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afaee3-bbdc-4990-8735-fa157cf8e55a admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:16.916065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afaee3-bbdc-4990-8735-fa157cf8e55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:16.916371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afaee3-bbdc-4990-8735-fa157cf8e55a admin admin] PATH_INFO: `/v3/projects/dc462a5ff5f649cdb9a24f2232855724/users/f884304cbb764a8ab4927c3ba8db892d/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:16.933651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afaee3-bbdc-4990-8735-fa157cf8e55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc462a5ff5f649cdb9a24f2232855724, user_id=f884304cbb764a8ab4927c3ba8db892d, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:16.936613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afaee3-bbdc-4990-8735-fa157cf8e55a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:16.962084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1212/2429] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:17:16 2022] PUT /identity/v3/projects/dc462a5ff5f649cdb9a24f2232855724/users/f884304cbb764a8ab4927c3ba8db892d/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:17:16.973103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724c71b-c657-4369-9242-8f70adbc9475 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:16.973435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724c71b-c657-4369-9242-8f70adbc9475 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:16.973919 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724c71b-c657-4369-9242-8f70adbc9475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:16.994474 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9724c71b-c657-4369-9242-8f70adbc9475 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:16.995769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9724c71b-c657-4369-9242-8f70adbc9475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:17.061175 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1218/2430] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:17.071108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afaee3-bbdc-4990-8735-fa157cf8e55a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:17.073052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b1afaee3-bbdc-4990-8735-fa157cf8e55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:17.149929 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afaee3-bbdc-4990-8735-fa157cf8e55a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:17.158251 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00a5a3-090a-4069-992f-c31d273df662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:17.159032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00a5a3-090a-4069-992f-c31d273df662 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:17.159325 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00a5a3-090a-4069-992f-c31d273df662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:17.159601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00a5a3-090a-4069-992f-c31d273df662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:17.171356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00a5a3-090a-4069-992f-c31d273df662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:17.172819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00a5a3-090a-4069-992f-c31d273df662 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:17.183852 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1213/2431] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:17:17.669769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185f63-e4f0-4505-baac-ecbd11e505b7 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:17.671234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185f63-e4f0-4505-baac-ecbd11e505b7 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:17.672181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185f63-e4f0-4505-baac-ecbd11e505b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:17.714822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d7185f63-e4f0-4505-baac-ecbd11e505b7 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:17.717946 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d7185f63-e4f0-4505-baac-ecbd11e505b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:17.884185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1219/2432] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:17:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:17.902497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00a5a3-090a-4069-992f-c31d273df662 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:17.904025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ce00a5a3-090a-4069-992f-c31d273df662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:18.020621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00a5a3-090a-4069-992f-c31d273df662 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:18.026865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a53f9-4029-4ea8-a24e-07f08cfa0124 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:18.028063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a53f9-4029-4ea8-a24e-07f08cfa0124 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:18.028524 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a53f9-4029-4ea8-a24e-07f08cfa0124 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:18.028959 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a53f9-4029-4ea8-a24e-07f08cfa0124 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:18.041586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a53f9-4029-4ea8-a24e-07f08cfa0124 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:18.043270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a53f9-4029-4ea8-a24e-07f08cfa0124 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:18.053864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1214/2433] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:17:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:17:19.272714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d482413-ad78-4432-8baa-1b0b50825af3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:19.273288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d482413-ad78-4432-8baa-1b0b50825af3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:19.273288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d482413-ad78-4432-8baa-1b0b50825af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:19.315063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0d482413-ad78-4432-8baa-1b0b50825af3 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:19.318238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0d482413-ad78-4432-8baa-1b0b50825af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:19.326109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1220/2434] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 4 09:17:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:19.353557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a53f9-4029-4ea8-a24e-07f08cfa0124 service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:19.370049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a53f9-4029-4ea8-a24e-07f08cfa0124 service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:19.380813 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30b46b-2003-4d63-b69d-3997f69b0abd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:19.381805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30b46b-2003-4d63-b69d-3997f69b0abd service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:19.382289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30b46b-2003-4d63-b69d-3997f69b0abd service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:19.382620 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30b46b-2003-4d63-b69d-3997f69b0abd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:19.396054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30b46b-2003-4d63-b69d-3997f69b0abd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:19.397976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30b46b-2003-4d63-b69d-3997f69b0abd service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:19.408894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1215/2435] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:17:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:17:19.880054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d048d-1a73-445d-8718-30d5c58019c5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:19.880349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d048d-1a73-445d-8718-30d5c58019c5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:19.880616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d048d-1a73-445d-8718-30d5c58019c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:19.917625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-188d048d-1a73-445d-8718-30d5c58019c5 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:19.923394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-188d048d-1a73-445d-8718-30d5c58019c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:19.943844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1221/2436] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:17:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:19.950639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30b46b-2003-4d63-b69d-3997f69b0abd service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:19.964379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30b46b-2003-4d63-b69d-3997f69b0abd service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:19.978996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced58efe-6036-438c-ab5b-4afa2a8b3ecf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:19.979678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced58efe-6036-438c-ab5b-4afa2a8b3ecf service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:19.979961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced58efe-6036-438c-ab5b-4afa2a8b3ecf service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:19.980236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced58efe-6036-438c-ab5b-4afa2a8b3ecf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:19.990531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced58efe-6036-438c-ab5b-4afa2a8b3ecf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:19.992175 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced58efe-6036-438c-ab5b-4afa2a8b3ecf service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:20.006316 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1216/2437] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:17:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:17:20.717413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64aa850-bbbb-4eba-bfc3-37069efa7e23 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:20.718006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64aa850-bbbb-4eba-bfc3-37069efa7e23 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:20.718374 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64aa850-bbbb-4eba-bfc3-37069efa7e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:20.753022 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c64aa850-bbbb-4eba-bfc3-37069efa7e23 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:20.755810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c64aa850-bbbb-4eba-bfc3-37069efa7e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:20.788130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1222/2438] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 4 09:17:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:20.791484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced58efe-6036-438c-ab5b-4afa2a8b3ecf service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:20.811051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced58efe-6036-438c-ab5b-4afa2a8b3ecf service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:20.828144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f634251-e29e-430d-b878-f5ffeaf19582 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:20.829437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f634251-e29e-430d-b878-f5ffeaf19582 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:20.829977 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f634251-e29e-430d-b878-f5ffeaf19582 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:20.830477 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f634251-e29e-430d-b878-f5ffeaf19582 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:20.849556 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f634251-e29e-430d-b878-f5ffeaf19582 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:20.851863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f634251-e29e-430d-b878-f5ffeaf19582 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:20.869442 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1217/2439] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:17:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:17:23.911784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099be3b1-0b62-4fc7-863a-f7fa92df6ce1 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:23.911784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099be3b1-0b62-4fc7-863a-f7fa92df6ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:23.912476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099be3b1-0b62-4fc7-863a-f7fa92df6ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:24.008876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-099be3b1-0b62-4fc7-863a-f7fa92df6ce1 None None] MFA Rules not processed for user `f884304cbb764a8ab4927c3ba8db892d`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:24.010348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-099be3b1-0b62-4fc7-863a-f7fa92df6ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:24.108414 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:24.211491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f634251-e29e-430d-b878-f5ffeaf19582 service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:24.215367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6f634251-e29e-430d-b878-f5ffeaf19582 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:24.273038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f634251-e29e-430d-b878-f5ffeaf19582 service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:24.279152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1223/2440] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:24.283917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099be3b1-0b62-4fc7-863a-f7fa92df6ce1 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:24.288525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020c1d0-51d3-4400-a8e2-024d8aa09af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:24.289604 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020c1d0-51d3-4400-a8e2-024d8aa09af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:24.289919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020c1d0-51d3-4400-a8e2-024d8aa09af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:24.290207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020c1d0-51d3-4400-a8e2-024d8aa09af5 admin admin] PATH_INFO: `/v3/users/a2435e1faabf48b0922de38e93215a48` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:24.292383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020c1d0-51d3-4400-a8e2-024d8aa09af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2435e1faabf48b0922de38e93215a48)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:24.294827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020c1d0-51d3-4400-a8e2-024d8aa09af5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:24.296668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099be3b1-0b62-4fc7-863a-f7fa92df6ce1 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:24.304722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75b327-49e4-4a17-b206-3fe50794edc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:24.305396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75b327-49e4-4a17-b206-3fe50794edc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:24.305683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75b327-49e4-4a17-b206-3fe50794edc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:24.305979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75b327-49e4-4a17-b206-3fe50794edc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:24.310188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75b327-49e4-4a17-b206-3fe50794edc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:24.312623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75b327-49e4-4a17-b206-3fe50794edc6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:24.318377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:24.379246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1224/2441] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:17:24 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:17:24.385297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9020c1d0-51d3-4400-a8e2-024d8aa09af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2435e1faabf48b0922de38e93215a48'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:24.387317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9020c1d0-51d3-4400-a8e2-024d8aa09af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2435e1faabf48b0922de38e93215a48'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:24.388206 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9020c1d0-51d3-4400-a8e2-024d8aa09af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2435e1faabf48b0922de38e93215a48'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:24.393187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75b327-49e4-4a17-b206-3fe50794edc6 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:24.401010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9020c1d0-51d3-4400-a8e2-024d8aa09af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2435e1faabf48b0922de38e93215a48'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:24.413136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1e75b327-49e4-4a17-b206-3fe50794edc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:24.415241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9020c1d0-51d3-4400-a8e2-024d8aa09af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2435e1faabf48b0922de38e93215a48'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:24.439190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1218/2442] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:17:24 2022] DELETE /identity/v3/users/a2435e1faabf48b0922de38e93215a48 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:17:24.441008 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb82d6-785f-4dcc-bce3-87772c1b6e26 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:24.441538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb82d6-785f-4dcc-bce3-87772c1b6e26 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:24.442021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb82d6-785f-4dcc-bce3-87772c1b6e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:24.465940 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0aeb82d6-785f-4dcc-bce3-87772c1b6e26 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:24.467657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0aeb82d6-785f-4dcc-bce3-87772c1b6e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:24.563592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1219/2443] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:24.572717 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb82d6-785f-4dcc-bce3-87772c1b6e26 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:24.584198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0aeb82d6-785f-4dcc-bce3-87772c1b6e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:24.620606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75b327-49e4-4a17-b206-3fe50794edc6 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:24.631470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d50198-7ce7-4e4c-a70c-9528a4430878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:24.632983 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d50198-7ce7-4e4c-a70c-9528a4430878 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:24.634031 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d50198-7ce7-4e4c-a70c-9528a4430878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:24.646253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d50198-7ce7-4e4c-a70c-9528a4430878 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:24.654253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d50198-7ce7-4e4c-a70c-9528a4430878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:24.657327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d50198-7ce7-4e4c-a70c-9528a4430878 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:24.725180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1225/2444] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:17:24 2022] POST /identity/v3/users => generated 296 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:17:24.729565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb82d6-785f-4dcc-bce3-87772c1b6e26 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:24.731221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d50198-7ce7-4e4c-a70c-9528a4430878 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:24.740905 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e03119-6384-4c41-b874-e6e17dfcd2fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:24.741579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e03119-6384-4c41-b874-e6e17dfcd2fe service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:24.741870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e03119-6384-4c41-b874-e6e17dfcd2fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:24.742167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e03119-6384-4c41-b874-e6e17dfcd2fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:24.747613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d50198-7ce7-4e4c-a70c-9528a4430878 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:24.759921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e03119-6384-4c41-b874-e6e17dfcd2fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:24.761743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e03119-6384-4c41-b874-e6e17dfcd2fe service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:24.767628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7436a0-bae2-4aa2-b7b9-df78542d5a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:24.768323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7436a0-bae2-4aa2-b7b9-df78542d5a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:24.768704 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7436a0-bae2-4aa2-b7b9-df78542d5a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:24.769038 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7436a0-bae2-4aa2-b7b9-df78542d5a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:24.769717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7436a0-bae2-4aa2-b7b9-df78542d5a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:24.771743 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7436a0-bae2-4aa2-b7b9-df78542d5a40 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:24.822610 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1220/2445] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:24.826420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1226/2446] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:17:24.830472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e03119-6384-4c41-b874-e6e17dfcd2fe service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:24.841983 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e03119-6384-4c41-b874-e6e17dfcd2fe service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:24.856174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6192b9-0abf-4d92-a54b-d6c97553e8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:24.857240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6192b9-0abf-4d92-a54b-d6c97553e8bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:24.857453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6192b9-0abf-4d92-a54b-d6c97553e8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:24.857789 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6192b9-0abf-4d92-a54b-d6c97553e8bc admin admin] PATH_INFO: `/v3/projects/3a47285027a241a6bf6e9aee8b5bde37/users/e814866b17714077bd0f6f501a49dbfb/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:24.929908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6192b9-0abf-4d92-a54b-d6c97553e8bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a47285027a241a6bf6e9aee8b5bde37, user_id=e814866b17714077bd0f6f501a49dbfb, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:24.933171 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6192b9-0abf-4d92-a54b-d6c97553e8bc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:24.963101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1221/2447] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:17:24 2022] PUT /identity/v3/projects/3a47285027a241a6bf6e9aee8b5bde37/users/e814866b17714077bd0f6f501a49dbfb/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:17:24.967491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7436a0-bae2-4aa2-b7b9-df78542d5a40 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:24.990134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7436a0-bae2-4aa2-b7b9-df78542d5a40 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:25.127000 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdc473-786d-400b-bbf4-79ee1d9b05ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:25.127756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdc473-786d-400b-bbf4-79ee1d9b05ab admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:25.128065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdc473-786d-400b-bbf4-79ee1d9b05ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:25.128344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdc473-786d-400b-bbf4-79ee1d9b05ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:25.129283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdc473-786d-400b-bbf4-79ee1d9b05ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:25.136708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdc473-786d-400b-bbf4-79ee1d9b05ab admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:25.165974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1227/2448] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:17:25.168400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6192b9-0abf-4d92-a54b-d6c97553e8bc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:25.187746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6192b9-0abf-4d92-a54b-d6c97553e8bc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:25.204950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb4c43-1d7d-4eaa-bc40-200565eff5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:25.205871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb4c43-1d7d-4eaa-bc40-200565eff5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:25.207143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb4c43-1d7d-4eaa-bc40-200565eff5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:25.208842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb4c43-1d7d-4eaa-bc40-200565eff5d5 admin admin] PATH_INFO: `/v3/projects/3a47285027a241a6bf6e9aee8b5bde37/users/e814866b17714077bd0f6f501a49dbfb/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:25.215295 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb4c43-1d7d-4eaa-bc40-200565eff5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a47285027a241a6bf6e9aee8b5bde37, user_id=e814866b17714077bd0f6f501a49dbfb, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:25.220044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb4c43-1d7d-4eaa-bc40-200565eff5d5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:25.234467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:25.274446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1222/2449] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:17:25 2022] PUT /identity/v3/projects/3a47285027a241a6bf6e9aee8b5bde37/users/e814866b17714077bd0f6f501a49dbfb/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:17:25.362663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdc473-786d-400b-bbf4-79ee1d9b05ab admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:25.364866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e9fdc473-786d-400b-bbf4-79ee1d9b05ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:25.385521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdc473-786d-400b-bbf4-79ee1d9b05ab admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:25.408862 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:25.539793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:25.541178 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:25.541476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:25.541757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] PATH_INFO: `/v3/projects/10fc941c0bca498e9d6cfeed12f0c6ef` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:25.543847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10fc941c0bca498e9d6cfeed12f0c6ef)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:25.546362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:25.633247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10fc941c0bca498e9d6cfeed12f0c6ef', 'request_id': 'req-43157d38-1ece-418d-9bfa-79cec5089ad1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:25.663315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10fc941c0bca498e9d6cfeed12f0c6ef', 'request_id': 'req-43157d38-1ece-418d-9bfa-79cec5089ad1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:25.675102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10fc941c0bca498e9d6cfeed12f0c6ef', 'request_id': 'req-43157d38-1ece-418d-9bfa-79cec5089ad1'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:25.724195 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] The token cache is being invalidate because project 10fc941c0bca498e9d6cfeed12f0c6ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:17:25.729871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:25.732092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:25.755440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1228/2450] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:17:25 2022] DELETE /identity/v3/projects/10fc941c0bca498e9d6cfeed12f0c6ef => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:17:32.580469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4e724-68e5-4a02-908b-556ed4f2a193 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:32.581503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4e724-68e5-4a02-908b-556ed4f2a193 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:32.582168 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4e724-68e5-4a02-908b-556ed4f2a193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:32.656127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f7e4e724-68e5-4a02-908b-556ed4f2a193 None None] MFA Rules not processed for user `e814866b17714077bd0f6f501a49dbfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:32.658349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f7e4e724-68e5-4a02-908b-556ed4f2a193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:32.803096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1223/2451] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:32.830395 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:32.832211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:32.977104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43157d38-1ece-418d-9bfa-79cec5089ad1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:32.983957 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16da92a-d94a-4452-9068-d4f20dbece6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:32.984733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16da92a-d94a-4452-9068-d4f20dbece6c service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:32.985197 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16da92a-d94a-4452-9068-d4f20dbece6c service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:32.985537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16da92a-d94a-4452-9068-d4f20dbece6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:32.987511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d16da92a-d94a-4452-9068-d4f20dbece6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:32.999307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:33.067575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16da92a-d94a-4452-9068-d4f20dbece6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:33.070081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16da92a-d94a-4452-9068-d4f20dbece6c service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:33.082290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1229/2452] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:42.798688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=36 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:43.077429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=22 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:43.750618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412883f-58e7-4601-867b-2aa92aa36ef8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:43.750917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412883f-58e7-4601-867b-2aa92aa36ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:43.751204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412883f-58e7-4601-867b-2aa92aa36ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:43.770316 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7412883f-58e7-4601-867b-2aa92aa36ef8 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:43.771540 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7412883f-58e7-4601-867b-2aa92aa36ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:43.829139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1224/2453] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 4 09:17:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:43.842717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16da92a-d94a-4452-9068-d4f20dbece6c service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:43.844316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d16da92a-d94a-4452-9068-d4f20dbece6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:43.918770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16da92a-d94a-4452-9068-d4f20dbece6c service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:43.924066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d392f3-4d4a-4453-bf66-a2224c11fc5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:43.924722 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d392f3-4d4a-4453-bf66-a2224c11fc5c service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:43.925008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d392f3-4d4a-4453-bf66-a2224c11fc5c service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:43.925283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d392f3-4d4a-4453-bf66-a2224c11fc5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:43.934103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d392f3-4d4a-4453-bf66-a2224c11fc5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:43.935708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d392f3-4d4a-4453-bf66-a2224c11fc5c service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:43.944155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1230/2454] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:17:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:17:45.945937 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412883f-58e7-4601-867b-2aa92aa36ef8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:45.947488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7412883f-58e7-4601-867b-2aa92aa36ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:46.047720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412883f-58e7-4601-867b-2aa92aa36ef8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:46.056053 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3b68d-89a4-4cf7-a154-b04214f98888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:46.057647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3b68d-89a4-4cf7-a154-b04214f98888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:46.058019 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3b68d-89a4-4cf7-a154-b04214f98888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:46.058373 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3b68d-89a4-4cf7-a154-b04214f98888 admin admin] PATH_INFO: `/v3/users/34edc297b377425b92683fc6530e8135` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:46.059816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3b68d-89a4-4cf7-a154-b04214f98888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34edc297b377425b92683fc6530e8135)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:46.062307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3b68d-89a4-4cf7-a154-b04214f98888 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:46.130705 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3ea3b68d-89a4-4cf7-a154-b04214f98888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34edc297b377425b92683fc6530e8135'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:46.131289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3ea3b68d-89a4-4cf7-a154-b04214f98888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34edc297b377425b92683fc6530e8135'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:46.132228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3ea3b68d-89a4-4cf7-a154-b04214f98888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34edc297b377425b92683fc6530e8135'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:46.141516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3ea3b68d-89a4-4cf7-a154-b04214f98888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34edc297b377425b92683fc6530e8135'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:46.149583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3ea3b68d-89a4-4cf7-a154-b04214f98888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34edc297b377425b92683fc6530e8135'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:46.172428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1225/2455] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:17:45 2022] DELETE /identity/v3/users/34edc297b377425b92683fc6530e8135 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:17:46.420780 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d392f3-4d4a-4453-bf66-a2224c11fc5c service swift] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:46.436223 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d392f3-4d4a-4453-bf66-a2224c11fc5c service swift] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:46.524925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:46.525816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:46.526129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:46.526662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] PATH_INFO: `/v3/projects/b7869946a7ac4ac7a15fab3dfe687abd` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:46.527874 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7869946a7ac4ac7a15fab3dfe687abd)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:46.529781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:46.595438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7869946a7ac4ac7a15fab3dfe687abd', 'request_id': 'req-5baa4c78-b60d-42a9-8859-4b78ec9808e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:46.603812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7869946a7ac4ac7a15fab3dfe687abd', 'request_id': 'req-5baa4c78-b60d-42a9-8859-4b78ec9808e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:46.611366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7869946a7ac4ac7a15fab3dfe687abd', 'request_id': 'req-5baa4c78-b60d-42a9-8859-4b78ec9808e6'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:46.623101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] The token cache is being invalidate because project b7869946a7ac4ac7a15fab3dfe687abd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:17:46.623389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:46.624170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-5baa4c78-b60d-42a9-8859-4b78ec9808e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:17:46.625726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1231/2456] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:17:46 2022] DELETE /identity/v3/projects/b7869946a7ac4ac7a15fab3dfe687abd => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:17:46.632658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c83e2e-023c-4319-b981-43190da096ed None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:46.633025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c83e2e-023c-4319-b981-43190da096ed None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:46.633255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c83e2e-023c-4319-b981-43190da096ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:46.670016 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-15c83e2e-023c-4319-b981-43190da096ed None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:46.671265 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-15c83e2e-023c-4319-b981-43190da096ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:46.682539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:46.742297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1226/2457] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:46.750143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f84b5-99fc-4bee-9ec4-e017cf9b982f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:46.750405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f84b5-99fc-4bee-9ec4-e017cf9b982f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:46.750718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f84b5-99fc-4bee-9ec4-e017cf9b982f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:46.769744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f62f84b5-99fc-4bee-9ec4-e017cf9b982f None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:46.770976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f62f84b5-99fc-4bee-9ec4-e017cf9b982f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:46.779729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1232/2458] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:46.784231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c83e2e-023c-4319-b981-43190da096ed None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:46.795234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c83e2e-023c-4319-b981-43190da096ed None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:46.800391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc57f15-77d0-4fc1-bd44-6717d5231ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:46.801163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57f15-77d0-4fc1-bd44-6717d5231ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:46.801460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57f15-77d0-4fc1-bd44-6717d5231ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:46.801740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57f15-77d0-4fc1-bd44-6717d5231ed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:46.802517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc57f15-77d0-4fc1-bd44-6717d5231ed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:17:46.802810 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc57f15-77d0-4fc1-bd44-6717d5231ed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:46.804437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc57f15-77d0-4fc1-bd44-6717d5231ed5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:46.816698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1227/2459] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:17:46.822123 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f84b5-99fc-4bee-9ec4-e017cf9b982f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:46.831853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f84b5-99fc-4bee-9ec4-e017cf9b982f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:46.842614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d08af-9a94-49bc-9173-cd6871fb1e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:46.843316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d08af-9a94-49bc-9173-cd6871fb1e24 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:46.845066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d08af-9a94-49bc-9173-cd6871fb1e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:46.845345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d08af-9a94-49bc-9173-cd6871fb1e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:46.846089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d08af-9a94-49bc-9173-cd6871fb1e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:46.849093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d08af-9a94-49bc-9173-cd6871fb1e24 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:46.860563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:46.883109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1233/2460] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:17:46 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:17:46.888038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc57f15-77d0-4fc1-bd44-6717d5231ed5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:46.899100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc57f15-77d0-4fc1-bd44-6717d5231ed5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:46.974448 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1b562-1dd1-48ca-a8de-99325b59fbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:46.975190 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1b562-1dd1-48ca-a8de-99325b59fbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:46.975404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1b562-1dd1-48ca-a8de-99325b59fbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:46.975672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1b562-1dd1-48ca-a8de-99325b59fbc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:46.976621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1b562-1dd1-48ca-a8de-99325b59fbc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:46.978979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1b562-1dd1-48ca-a8de-99325b59fbc2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:47.002735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1228/2461] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:17:46 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:17:47.007955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d08af-9a94-49bc-9173-cd6871fb1e24 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:47.019580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d08af-9a94-49bc-9173-cd6871fb1e24 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:47.025312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c52815-d192-44b0-9642-fead4c6c354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:47.026119 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c52815-d192-44b0-9642-fead4c6c354a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:47.026401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c52815-d192-44b0-9642-fead4c6c354a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:47.026677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c52815-d192-44b0-9642-fead4c6c354a admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:47.027431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c52815-d192-44b0-9642-fead4c6c354a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:47.029040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c52815-d192-44b0-9642-fead4c6c354a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:47.040744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1234/2462] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:17:47.044332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1b562-1dd1-48ca-a8de-99325b59fbc2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:47.054771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1b562-1dd1-48ca-a8de-99325b59fbc2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:47.059778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8867bb-e0c7-4605-b107-76424769baf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:47.060499 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8867bb-e0c7-4605-b107-76424769baf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:47.060914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8867bb-e0c7-4605-b107-76424769baf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:47.061202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8867bb-e0c7-4605-b107-76424769baf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:47.062014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8867bb-e0c7-4605-b107-76424769baf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:47.063636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8867bb-e0c7-4605-b107-76424769baf4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:47.078020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1229/2463] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:17:47.079356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c52815-d192-44b0-9642-fead4c6c354a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:47.092058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c52815-d192-44b0-9642-fead4c6c354a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:47.096942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea29ab-befd-43ac-894d-161d03f983da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:47.097902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea29ab-befd-43ac-894d-161d03f983da admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:47.098438 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea29ab-befd-43ac-894d-161d03f983da admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:47.098970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea29ab-befd-43ac-894d-161d03f983da admin admin] PATH_INFO: `/v3/projects/670e51952bc24ea08f35d18f577e176a/users/7972dcd3aa1146779d9960bead16e1dc/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:47.117386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea29ab-befd-43ac-894d-161d03f983da admin admin] RBAC: Authorizing `identity:create_grant(project_id=670e51952bc24ea08f35d18f577e176a, user_id=7972dcd3aa1146779d9960bead16e1dc, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:47.120188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea29ab-befd-43ac-894d-161d03f983da admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:47.136988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1235/2464] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:17:47 2022] PUT /identity/v3/projects/670e51952bc24ea08f35d18f577e176a/users/7972dcd3aa1146779d9960bead16e1dc/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:17:47.147490 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceca2f-0223-4855-bfa7-d3d5f56d325f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:47.147783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceca2f-0223-4855-bfa7-d3d5f56d325f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:47.148060 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceca2f-0223-4855-bfa7-d3d5f56d325f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:47.179869 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-90ceca2f-0223-4855-bfa7-d3d5f56d325f None None] MFA Rules not processed for user `7972dcd3aa1146779d9960bead16e1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:47.181099 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-90ceca2f-0223-4855-bfa7-d3d5f56d325f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:47.248051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1230/2465] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:17:47.260681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea29ab-befd-43ac-894d-161d03f983da admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:47.263111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f6ea29ab-befd-43ac-894d-161d03f983da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:47.353491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea29ab-befd-43ac-894d-161d03f983da admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:47.357405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6ec0d-8ffe-487b-b3aa-14dbcb542b18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:47.358163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6ec0d-8ffe-487b-b3aa-14dbcb542b18 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:47.358395 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6ec0d-8ffe-487b-b3aa-14dbcb542b18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:47.358641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6ec0d-8ffe-487b-b3aa-14dbcb542b18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:47.370346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6ec0d-8ffe-487b-b3aa-14dbcb542b18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:47.371838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6ec0d-8ffe-487b-b3aa-14dbcb542b18 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:47.382662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1236/2466] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:51.835395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceca2f-0223-4855-bfa7-d3d5f56d325f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:17:51.837172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-90ceca2f-0223-4855-bfa7-d3d5f56d325f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:51.851684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:51.945367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceca2f-0223-4855-bfa7-d3d5f56d325f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:17:51.951050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3f5ad-c7c6-4f7e-adcd-73a0925240cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:17:51.951721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3f5ad-c7c6-4f7e-adcd-73a0925240cd service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:51.952006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3f5ad-c7c6-4f7e-adcd-73a0925240cd service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:51.952284 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3f5ad-c7c6-4f7e-adcd-73a0925240cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:51.954215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4eb3f5ad-c7c6-4f7e-adcd-73a0925240cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:51.968298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3f5ad-c7c6-4f7e-adcd-73a0925240cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:17:51.970034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3f5ad-c7c6-4f7e-adcd-73a0925240cd service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:17:51.982496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1231/2467] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:17:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:17:57.378536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:17:59.216841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8766cc-5bce-427e-aa8e-c8ea9ed0b0fe None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:17:59.217513 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8766cc-5bce-427e-aa8e-c8ea9ed0b0fe None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:17:59.217513 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8766cc-5bce-427e-aa8e-c8ea9ed0b0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:17:59.236418 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5c8766cc-5bce-427e-aa8e-c8ea9ed0b0fe None None] MFA Rules not processed for user `c0ebb8aa3b7146e4a34a565969154f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:17:59.237613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5c8766cc-5bce-427e-aa8e-c8ea9ed0b0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:17:59.307495 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1237/2468] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:17:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:01.975963 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=36 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:09.058186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21616-f04e-4b52-8050-78dd811e321b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:09.058846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21616-f04e-4b52-8050-78dd811e321b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:09.058846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21616-f04e-4b52-8050-78dd811e321b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:09.078376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-32a21616-f04e-4b52-8050-78dd811e321b None None] MFA Rules not processed for user `f884304cbb764a8ab4927c3ba8db892d`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:09.079607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-32a21616-f04e-4b52-8050-78dd811e321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:09.154677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1232/2469] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:18:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:09.298482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=25 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:12.087505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8766cc-5bce-427e-aa8e-c8ea9ed0b0fe None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:12.159884 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8766cc-5bce-427e-aa8e-c8ea9ed0b0fe None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:12.165235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:12.166159 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:12.166442 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:12.166713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] PATH_INFO: `/v3/users/7972dcd3aa1146779d9960bead16e1dc` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:12.167955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7972dcd3aa1146779d9960bead16e1dc)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:12.170376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:12.228909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7972dcd3aa1146779d9960bead16e1dc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:18:12.230426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7972dcd3aa1146779d9960bead16e1dc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:18:12.239200 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7972dcd3aa1146779d9960bead16e1dc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:18:12.247858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7972dcd3aa1146779d9960bead16e1dc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:18:12.248676 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7972dcd3aa1146779d9960bead16e1dc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:18:12.269993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1238/2470] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:18:12 2022] DELETE /identity/v3/users/7972dcd3aa1146779d9960bead16e1dc => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:18:12.274553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef140f14-fcfd-4613-b3be-350cf320cb01 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:12.275081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef140f14-fcfd-4613-b3be-350cf320cb01 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:12.275081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef140f14-fcfd-4613-b3be-350cf320cb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:12.292806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ef140f14-fcfd-4613-b3be-350cf320cb01 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:12.294495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ef140f14-fcfd-4613-b3be-350cf320cb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:12.355935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1233/2471] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:12.366122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:12.369306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:12.444130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a63343-a866-4d69-a9cc-cdeef7da3235 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:12.448152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08207408-3401-4bbc-8b80-2681a914a9fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:12.448807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08207408-3401-4bbc-8b80-2681a914a9fc service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:12.449088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08207408-3401-4bbc-8b80-2681a914a9fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:12.449362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08207408-3401-4bbc-8b80-2681a914a9fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:12.459972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08207408-3401-4bbc-8b80-2681a914a9fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:12.461765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08207408-3401-4bbc-8b80-2681a914a9fc service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:12.473327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1239/2472] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:18:12.788240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef140f14-fcfd-4613-b3be-350cf320cb01 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:12.789735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ef140f14-fcfd-4613-b3be-350cf320cb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:12.815400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef140f14-fcfd-4613-b3be-350cf320cb01 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:12.841614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:12.843944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:12.844317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:12.844608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] PATH_INFO: `/v3/projects/670e51952bc24ea08f35d18f577e176a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:12.852501 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=670e51952bc24ea08f35d18f577e176a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:12.854452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:12.918883 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670e51952bc24ea08f35d18f577e176a', 'request_id': 'req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:18:12.931006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670e51952bc24ea08f35d18f577e176a', 'request_id': 'req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:18:12.940371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670e51952bc24ea08f35d18f577e176a', 'request_id': 'req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:18:12.951818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] The token cache is being invalidate because project 670e51952bc24ea08f35d18f577e176a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:18:12.952143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:18:12.952779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-091f94d7-916a-420d-a83d-c0e8b6e4d5b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:18:12.954654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1234/2473] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:18:12 2022] DELETE /identity/v3/projects/670e51952bc24ea08f35d18f577e176a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:18:12.959956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae277be-1b89-4480-bff1-ed43811794a8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:12.960254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae277be-1b89-4480-bff1-ed43811794a8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:12.960593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae277be-1b89-4480-bff1-ed43811794a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:12.980579 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-cae277be-1b89-4480-bff1-ed43811794a8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:12.981832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cae277be-1b89-4480-bff1-ed43811794a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:13.022127 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:13.067236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1240/2474] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:13.075216 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa9e67-d3a5-4c10-a6d9-4672cb0714a5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:13.075542 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa9e67-d3a5-4c10-a6d9-4672cb0714a5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:13.075826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa9e67-d3a5-4c10-a6d9-4672cb0714a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:13.095215 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3efa9e67-d3a5-4c10-a6d9-4672cb0714a5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:13.096638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3efa9e67-d3a5-4c10-a6d9-4672cb0714a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:13.105181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1235/2475] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:13.109561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae277be-1b89-4480-bff1-ed43811794a8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:13.120029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae277be-1b89-4480-bff1-ed43811794a8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:13.124701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc2fcf-486f-418a-a806-8741693cec81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:13.125672 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc2fcf-486f-418a-a806-8741693cec81 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:13.125814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc2fcf-486f-418a-a806-8741693cec81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:13.126072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc2fcf-486f-418a-a806-8741693cec81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:13.126929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc2fcf-486f-418a-a806-8741693cec81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:18:13.127242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc2fcf-486f-418a-a806-8741693cec81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:13.128955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc2fcf-486f-418a-a806-8741693cec81 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:13.143084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1241/2476] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:18:13.147301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa9e67-d3a5-4c10-a6d9-4672cb0714a5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:13.157018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa9e67-d3a5-4c10-a6d9-4672cb0714a5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:13.162227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e3b76-8b5c-4de8-94dd-c4c90dee283f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:13.163091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e3b76-8b5c-4de8-94dd-c4c90dee283f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:13.163406 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e3b76-8b5c-4de8-94dd-c4c90dee283f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:13.163709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e3b76-8b5c-4de8-94dd-c4c90dee283f admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:13.164440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e3b76-8b5c-4de8-94dd-c4c90dee283f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:13.166431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e3b76-8b5c-4de8-94dd-c4c90dee283f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:13.187405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:13.198150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1236/2477] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:18:13 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:13.199343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc2fcf-486f-418a-a806-8741693cec81 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:13.211341 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc2fcf-486f-418a-a806-8741693cec81 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:13.274020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4189a-33f2-4091-97fc-5a0b34334e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:13.274881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4189a-33f2-4091-97fc-5a0b34334e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:13.275203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4189a-33f2-4091-97fc-5a0b34334e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:13.275578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4189a-33f2-4091-97fc-5a0b34334e0e admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:13.276581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4189a-33f2-4091-97fc-5a0b34334e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:13.279396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4189a-33f2-4091-97fc-5a0b34334e0e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:13.305576 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1242/2478] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:18:13 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:13.309914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e3b76-8b5c-4de8-94dd-c4c90dee283f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:13.321298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e3b76-8b5c-4de8-94dd-c4c90dee283f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:13.326935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bef0f-8b7d-4fb9-8162-bb72ebbab134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:13.327622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bef0f-8b7d-4fb9-8162-bb72ebbab134 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:13.327908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bef0f-8b7d-4fb9-8162-bb72ebbab134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:13.328187 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bef0f-8b7d-4fb9-8162-bb72ebbab134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:13.328917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bef0f-8b7d-4fb9-8162-bb72ebbab134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:13.330687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bef0f-8b7d-4fb9-8162-bb72ebbab134 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:13.342826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1237/2479] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:13.346923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4189a-33f2-4091-97fc-5a0b34334e0e admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:13.358417 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4189a-33f2-4091-97fc-5a0b34334e0e admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:13.367115 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c35e83-d4de-4ff7-975e-472a2f1230d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:13.368119 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c35e83-d4de-4ff7-975e-472a2f1230d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:13.368502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c35e83-d4de-4ff7-975e-472a2f1230d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:13.368941 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c35e83-d4de-4ff7-975e-472a2f1230d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:13.370155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c35e83-d4de-4ff7-975e-472a2f1230d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:13.372710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c35e83-d4de-4ff7-975e-472a2f1230d3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:13.385872 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1243/2480] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:13.391344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bef0f-8b7d-4fb9-8162-bb72ebbab134 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:13.408729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bef0f-8b7d-4fb9-8162-bb72ebbab134 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:13.415554 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e8605-99a6-486c-ae40-513705964aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:13.416570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e8605-99a6-486c-ae40-513705964aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:13.416870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e8605-99a6-486c-ae40-513705964aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:13.417324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e8605-99a6-486c-ae40-513705964aeb admin admin] PATH_INFO: `/v3/projects/8a4eaafcb5814b92a0c740931fcf422d/users/a9dfef8fef8d45d99636579271c71d51/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:13.434120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e8605-99a6-486c-ae40-513705964aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a4eaafcb5814b92a0c740931fcf422d, user_id=a9dfef8fef8d45d99636579271c71d51, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:13.436774 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e8605-99a6-486c-ae40-513705964aeb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:13.455370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1238/2481] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:18:13 2022] PUT /identity/v3/projects/8a4eaafcb5814b92a0c740931fcf422d/users/a9dfef8fef8d45d99636579271c71d51/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:18:13.461373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57155702-127e-447c-b01f-5a0cb796dd45 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:13.461658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57155702-127e-447c-b01f-5a0cb796dd45 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:13.461959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57155702-127e-447c-b01f-5a0cb796dd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:13.479891 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-57155702-127e-447c-b01f-5a0cb796dd45 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:13.481098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-57155702-127e-447c-b01f-5a0cb796dd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:13.543772 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1244/2482] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:13.553497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e8605-99a6-486c-ae40-513705964aeb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:13.555355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-621e8605-99a6-486c-ae40-513705964aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:13.621234 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e8605-99a6-486c-ae40-513705964aeb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:13.635992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905a6e8-8d4d-488f-9b42-676c0cf02b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:13.636723 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905a6e8-8d4d-488f-9b42-676c0cf02b19 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:13.637313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905a6e8-8d4d-488f-9b42-676c0cf02b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:13.637615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905a6e8-8d4d-488f-9b42-676c0cf02b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:13.655588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905a6e8-8d4d-488f-9b42-676c0cf02b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:13.657279 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905a6e8-8d4d-488f-9b42-676c0cf02b19 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:13.668467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1239/2483] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:18:18.693622 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5de9-f366-460a-885a-71c11c1d27be None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:18.694199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5de9-f366-460a-885a-71c11c1d27be None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:18.694199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5de9-f366-460a-885a-71c11c1d27be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:18.726624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-bd1c5de9-f366-460a-885a-71c11c1d27be None None] MFA Rules not processed for user `a9dfef8fef8d45d99636579271c71d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:18.728562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bd1c5de9-f366-460a-885a-71c11c1d27be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:18.747953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:18.804455 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1245/2484] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:18:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:18.808939 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905a6e8-8d4d-488f-9b42-676c0cf02b19 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:18.819468 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905a6e8-8d4d-488f-9b42-676c0cf02b19 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:18.824508 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57433862-fb4b-4b26-b073-d922722cc83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:18.825181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57433862-fb4b-4b26-b073-d922722cc83f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:18.825459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57433862-fb4b-4b26-b073-d922722cc83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:18.825938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57433862-fb4b-4b26-b073-d922722cc83f admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:18.826799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57433862-fb4b-4b26-b073-d922722cc83f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:18.828887 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57433862-fb4b-4b26-b073-d922722cc83f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:18.863449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1240/2485] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:18:18 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:18:18.864265 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c5de9-f366-460a-885a-71c11c1d27be None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:18.875587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c5de9-f366-460a-885a-71c11c1d27be None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:18.946851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40370ed-b021-4e5a-bf90-a86da061f5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:18.947627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40370ed-b021-4e5a-bf90-a86da061f5f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:18.948321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40370ed-b021-4e5a-bf90-a86da061f5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:18.948613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40370ed-b021-4e5a-bf90-a86da061f5f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:18.949329 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40370ed-b021-4e5a-bf90-a86da061f5f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:18.951458 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40370ed-b021-4e5a-bf90-a86da061f5f4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:18.975517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1246/2486] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:18:18 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:18:18.980128 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57433862-fb4b-4b26-b073-d922722cc83f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:18.994657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57433862-fb4b-4b26-b073-d922722cc83f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:19.000269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f10c3-a0d6-4a6e-ad18-f2c10b63c542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:19.001028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f10c3-a0d6-4a6e-ad18-f2c10b63c542 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:19.001381 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f10c3-a0d6-4a6e-ad18-f2c10b63c542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:19.001727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f10c3-a0d6-4a6e-ad18-f2c10b63c542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:19.002786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f10c3-a0d6-4a6e-ad18-f2c10b63c542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:19.005019 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f10c3-a0d6-4a6e-ad18-f2c10b63c542 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:19.028931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1241/2487] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:19.030102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40370ed-b021-4e5a-bf90-a86da061f5f4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:19.042779 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40370ed-b021-4e5a-bf90-a86da061f5f4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:19.047905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81409435-2ebc-42e1-93f1-c9bedbbeb69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:19.049180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81409435-2ebc-42e1-93f1-c9bedbbeb69f admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:19.049315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81409435-2ebc-42e1-93f1-c9bedbbeb69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:19.049505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81409435-2ebc-42e1-93f1-c9bedbbeb69f admin admin] PATH_INFO: `/v3/projects/a110bfdb567344a592e26dca6f9a7d12/users/7ddbac08c11747da9b343135b4eb438c/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:19.077578 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81409435-2ebc-42e1-93f1-c9bedbbeb69f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a110bfdb567344a592e26dca6f9a7d12, user_id=7ddbac08c11747da9b343135b4eb438c, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:19.080470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81409435-2ebc-42e1-93f1-c9bedbbeb69f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:19.103265 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1247/2488] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:18:19 2022] PUT /identity/v3/projects/a110bfdb567344a592e26dca6f9a7d12/users/7ddbac08c11747da9b343135b4eb438c/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:18:19.107546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f10c3-a0d6-4a6e-ad18-f2c10b63c542 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:19.118258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f10c3-a0d6-4a6e-ad18-f2c10b63c542 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:19.185595 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a6e9a-c460-4999-87cb-41fb895cdf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:19.186376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a6e9a-c460-4999-87cb-41fb895cdf05 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:19.186666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a6e9a-c460-4999-87cb-41fb895cdf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:19.186943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a6e9a-c460-4999-87cb-41fb895cdf05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:19.187674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a6e9a-c460-4999-87cb-41fb895cdf05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:19.189280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a6e9a-c460-4999-87cb-41fb895cdf05 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:19.201115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1242/2489] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:18:19.205276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81409435-2ebc-42e1-93f1-c9bedbbeb69f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:19.216639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81409435-2ebc-42e1-93f1-c9bedbbeb69f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:19.221533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30e5a9-94d0-4813-a47d-489d407971ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:19.222460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30e5a9-94d0-4813-a47d-489d407971ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:19.222799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30e5a9-94d0-4813-a47d-489d407971ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:19.223078 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30e5a9-94d0-4813-a47d-489d407971ac admin admin] PATH_INFO: `/v3/projects/a110bfdb567344a592e26dca6f9a7d12/users/7ddbac08c11747da9b343135b4eb438c/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:19.225159 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30e5a9-94d0-4813-a47d-489d407971ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a110bfdb567344a592e26dca6f9a7d12, user_id=7ddbac08c11747da9b343135b4eb438c, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:19.227860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30e5a9-94d0-4813-a47d-489d407971ac admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:19.239939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1248/2490] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:18:19 2022] PUT /identity/v3/projects/a110bfdb567344a592e26dca6f9a7d12/users/7ddbac08c11747da9b343135b4eb438c/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:18:24.157950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b2780-c31a-45d4-a8ec-d6a1d2e6b3cc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:24.158446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b2780-c31a-45d4-a8ec-d6a1d2e6b3cc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:24.158533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b2780-c31a-45d4-a8ec-d6a1d2e6b3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:24.185791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8e2b2780-c31a-45d4-a8ec-d6a1d2e6b3cc None None] MFA Rules not processed for user `7ddbac08c11747da9b343135b4eb438c`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:24.186981 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8e2b2780-c31a-45d4-a8ec-d6a1d2e6b3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:24.210479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:24.267690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1243/2491] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:18:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:24.280745 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30e5a9-94d0-4813-a47d-489d407971ac admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:24.282584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0e30e5a9-94d0-4813-a47d-489d407971ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:24.378823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30e5a9-94d0-4813-a47d-489d407971ac admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:24.383683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c34fa5-3d14-41d2-92ec-64e7861c05c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a90e384504e8d80264d6bf20f1d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:24.384520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c34fa5-3d14-41d2-92ec-64e7861c05c2 service glance] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:24.384880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c34fa5-3d14-41d2-92ec-64e7861c05c2 service glance] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:24.385226 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c34fa5-3d14-41d2-92ec-64e7861c05c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:24.390917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:24.466828 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c34fa5-3d14-41d2-92ec-64e7861c05c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:24.468316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c34fa5-3d14-41d2-92ec-64e7861c05c2 service glance] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:24.494054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1249/2492] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:18:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:18:24.666743 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cad2c-6a8b-4f57-9b05-63753375592b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:24.667042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cad2c-6a8b-4f57-9b05-63753375592b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:24.667329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cad2c-6a8b-4f57-9b05-63753375592b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:24.731318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-269cad2c-6a8b-4f57-9b05-63753375592b None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:24.732698 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-269cad2c-6a8b-4f57-9b05-63753375592b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:24.804855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1244/2493] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:18:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:18:24.828171 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c34fa5-3d14-41d2-92ec-64e7861c05c2 service glance] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:24.829698 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-54c34fa5-3d14-41d2-92ec-64e7861c05c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:24.925938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c34fa5-3d14-41d2-92ec-64e7861c05c2 service glance] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:24.931520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ff785-c676-4536-b61c-6bcd90da3311 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:24.932281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ff785-c676-4536-b61c-6bcd90da3311 service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:24.932561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ff785-c676-4536-b61c-6bcd90da3311 service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:24.932833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ff785-c676-4536-b61c-6bcd90da3311 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:24.942437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ff785-c676-4536-b61c-6bcd90da3311 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:24.944021 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ff785-c676-4536-b61c-6bcd90da3311 service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:24.952964 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1250/2494] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:18:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:18:32.499309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cad2c-6a8b-4f57-9b05-63753375592b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:32.500767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-269cad2c-6a8b-4f57-9b05-63753375592b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:32.570711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cad2c-6a8b-4f57-9b05-63753375592b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:32.576785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be562252-4c18-4c22-bea2-1144134f46bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:32.577238 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be562252-4c18-4c22-bea2-1144134f46bb service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:32.577530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be562252-4c18-4c22-bea2-1144134f46bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:32.577810 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be562252-4c18-4c22-bea2-1144134f46bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:32.583543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-be562252-4c18-4c22-bea2-1144134f46bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:32.634716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:32.662913 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be562252-4c18-4c22-bea2-1144134f46bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:32.673952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be562252-4c18-4c22-bea2-1144134f46bb service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:32.681612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1245/2495] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:18:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:18:34.949282 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=66 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:35.119525 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b927b1-df07-4dd5-98cc-1aa5f7b19ab0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:35.122810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b927b1-df07-4dd5-98cc-1aa5f7b19ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:35.123109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b927b1-df07-4dd5-98cc-1aa5f7b19ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:35.148351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a2b927b1-df07-4dd5-98cc-1aa5f7b19ab0 None None] MFA Rules not processed for user `a9dfef8fef8d45d99636579271c71d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:35.152877 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a2b927b1-df07-4dd5-98cc-1aa5f7b19ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:35.159127 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1251/2496] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:18:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:18:39.392010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d37d8b-089a-4d3b-b6fa-614a96d9a0b3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:39.392575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d37d8b-089a-4d3b-b6fa-614a96d9a0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:39.392575 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d37d8b-089a-4d3b-b6fa-614a96d9a0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:39.412247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-72d37d8b-089a-4d3b-b6fa-614a96d9a0b3 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:39.413518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-72d37d8b-089a-4d3b-b6fa-614a96d9a0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:39.422631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1246/2497] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:18:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:18:39.441119 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b927b1-df07-4dd5-98cc-1aa5f7b19ab0 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:18:39.456875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b927b1-df07-4dd5-98cc-1aa5f7b19ab0 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:18:39.463929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25e190-f54b-4366-843d-f51b99803df4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:18:39.465055 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e190-f54b-4366-843d-f51b99803df4 service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:39.465542 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e190-f54b-4366-843d-f51b99803df4 service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:39.465896 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e190-f54b-4366-843d-f51b99803df4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:39.476557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25e190-f54b-4366-843d-f51b99803df4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:18:39.478316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25e190-f54b-4366-843d-f51b99803df4 service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:18:39.488053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1252/2498] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:18:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:18:49.406354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=13 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:49.484518 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=9 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:18:58.522727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3402-7d4e-4302-86c3-7533b3d9f7cf None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:18:58.523646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3402-7d4e-4302-86c3-7533b3d9f7cf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:18:58.524104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3402-7d4e-4302-86c3-7533b3d9f7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:18:58.544231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-abcb3402-7d4e-4302-86c3-7533b3d9f7cf None None] MFA Rules not processed for user `1e916bf18efe421895c8d7b28fcd55d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:18:58.545446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-abcb3402-7d4e-4302-86c3-7533b3d9f7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:18:58.602894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1247/2499] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:18:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:08.595512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=25 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:12.304479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25e190-f54b-4366-843d-f51b99803df4 service swift] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:12.314574 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25e190-f54b-4366-843d-f51b99803df4 service swift] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:12.319245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0e598-d109-4fb4-a2f1-04f6ef38fcc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:12.319889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0e598-d109-4fb4-a2f1-04f6ef38fcc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:12.320171 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0e598-d109-4fb4-a2f1-04f6ef38fcc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:12.320449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0e598-d109-4fb4-a2f1-04f6ef38fcc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:12.329944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0e598-d109-4fb4-a2f1-04f6ef38fcc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:12.331395 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0e598-d109-4fb4-a2f1-04f6ef38fcc2 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:12.340870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1253/2500] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:14.198970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb3402-7d4e-4302-86c3-7533b3d9f7cf None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:14.200712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-abcb3402-7d4e-4302-86c3-7533b3d9f7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:14.277456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb3402-7d4e-4302-86c3-7533b3d9f7cf None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:14.283644 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:14.284894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:14.285236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:14.285522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] PATH_INFO: `/v3/users/c0ebb8aa3b7146e4a34a565969154f9a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:14.287700 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0ebb8aa3b7146e4a34a565969154f9a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:14.292543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:14.396647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0ebb8aa3b7146e4a34a565969154f9a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:14.397527 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0ebb8aa3b7146e4a34a565969154f9a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:14.398137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0ebb8aa3b7146e4a34a565969154f9a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:14.408953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0ebb8aa3b7146e4a34a565969154f9a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:14.423890 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0ebb8aa3b7146e4a34a565969154f9a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:14.451035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1248/2501] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:19:14 2022] DELETE /identity/v3/users/c0ebb8aa3b7146e4a34a565969154f9a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:14.707868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0e598-d109-4fb4-a2f1-04f6ef38fcc2 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:14.764906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0e598-d109-4fb4-a2f1-04f6ef38fcc2 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:14.862163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:14.863621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:14.863965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:14.864313 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] PATH_INFO: `/v3/projects/fec91e0c883943448c9d0e8d4d5e09ae` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:14.865598 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec91e0c883943448c9d0e8d4d5e09ae)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:14.867713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:14.926934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec91e0c883943448c9d0e8d4d5e09ae', 'request_id': 'req-8d33cf5f-5876-4e03-a28a-199962f86f45'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:14.937170 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec91e0c883943448c9d0e8d4d5e09ae', 'request_id': 'req-8d33cf5f-5876-4e03-a28a-199962f86f45'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:14.944750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec91e0c883943448c9d0e8d4d5e09ae', 'request_id': 'req-8d33cf5f-5876-4e03-a28a-199962f86f45'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:14.959335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] The token cache is being invalidate because project fec91e0c883943448c9d0e8d4d5e09ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:19:14.959796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:14.961084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:19:14.979307 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1254/2502] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:19:14 2022] DELETE /identity/v3/projects/fec91e0c883943448c9d0e8d4d5e09ae => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:19:24.446589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:24.954988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:19:45.453920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:45.455538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:45.518932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd158973-ffd6-4af9-a307-d7b07c0f48cb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:45.578307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863b310-2b0e-49b0-ae7f-8d7e94ce7ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:45.578969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863b310-2b0e-49b0-ae7f-8d7e94ce7ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:45.579248 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863b310-2b0e-49b0-ae7f-8d7e94ce7ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:45.579528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863b310-2b0e-49b0-ae7f-8d7e94ce7ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:45.585537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1863b310-2b0e-49b0-ae7f-8d7e94ce7ad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:45.666818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863b310-2b0e-49b0-ae7f-8d7e94ce7ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:45.667988 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863b310-2b0e-49b0-ae7f-8d7e94ce7ad6 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:45.681681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1249/2503] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:45.856961 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:45.859176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:45.872669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33cf5f-5876-4e03-a28a-199962f86f45 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:45.876376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa26e91-8897-41c5-8eff-924b36f8eaf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:45.877028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa26e91-8897-41c5-8eff-924b36f8eaf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:45.877521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa26e91-8897-41c5-8eff-924b36f8eaf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:45.877804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa26e91-8897-41c5-8eff-924b36f8eaf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:45.879699 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7fa26e91-8897-41c5-8eff-924b36f8eaf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:45.895266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa26e91-8897-41c5-8eff-924b36f8eaf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:45.896732 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa26e91-8897-41c5-8eff-924b36f8eaf4 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:45.909768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1255/2504] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:48.088456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14941a05-1869-4d25-9143-bf470af86c7b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:48.088951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14941a05-1869-4d25-9143-bf470af86c7b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:48.089068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14941a05-1869-4d25-9143-bf470af86c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:48.106538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-14941a05-1869-4d25-9143-bf470af86c7b None None] MFA Rules not processed for user `43bcd8fb25604011bfd1df823ea4ddad`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:19:48.107740 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-14941a05-1869-4d25-9143-bf470af86c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:48.169662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1250/2505] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:19:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:19:54.600439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa26e91-8897-41c5-8eff-924b36f8eaf4 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:19:54.602617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7fa26e91-8897-41c5-8eff-924b36f8eaf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:54.679132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa26e91-8897-41c5-8eff-924b36f8eaf4 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:19:54.693267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ea752-d347-475a-a8cf-f4af8d8b33af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:19:54.693863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ea752-d347-475a-a8cf-f4af8d8b33af service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:19:54.694172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ea752-d347-475a-a8cf-f4af8d8b33af service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:19:54.694498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ea752-d347-475a-a8cf-f4af8d8b33af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:19:54.697400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9d7ea752-d347-475a-a8cf-f4af8d8b33af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:19:54.771066 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ea752-d347-475a-a8cf-f4af8d8b33af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:19:54.773327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ea752-d347-475a-a8cf-f4af8d8b33af service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:19:54.782818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1256/2506] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:19:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:19:58.163089 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:00.228603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14941a05-1869-4d25-9143-bf470af86c7b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:00.230309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-14941a05-1869-4d25-9143-bf470af86c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:00.247685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14941a05-1869-4d25-9143-bf470af86c7b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:00.254456 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7c900-6317-4725-a15e-f41214a9e271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:00.255129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7c900-6317-4725-a15e-f41214a9e271 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:00.255415 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7c900-6317-4725-a15e-f41214a9e271 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:00.255669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7c900-6317-4725-a15e-f41214a9e271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:00.257696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bff7c900-6317-4725-a15e-f41214a9e271 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:00.269435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7c900-6317-4725-a15e-f41214a9e271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:00.271071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7c900-6317-4725-a15e-f41214a9e271 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:00.281000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1251/2507] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:04.778599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=54 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:06.622600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ea752-d347-475a-a8cf-f4af8d8b33af service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:06.634008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ea752-d347-475a-a8cf-f4af8d8b33af service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:06.637942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c5433-bb82-4b00-9595-7af80ed395ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:06.639436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c5433-bb82-4b00-9595-7af80ed395ae service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:06.639635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c5433-bb82-4b00-9595-7af80ed395ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:06.640188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c5433-bb82-4b00-9595-7af80ed395ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:06.642582 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5a8c5433-bb82-4b00-9595-7af80ed395ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:06.653822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c5433-bb82-4b00-9595-7af80ed395ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:06.655321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c5433-bb82-4b00-9595-7af80ed395ae service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:06.664781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1257/2508] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:06.797666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50835-0835-49f1-be14-f165cca6838e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:06.798280 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50835-0835-49f1-be14-f165cca6838e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:06.798374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50835-0835-49f1-be14-f165cca6838e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:06.803925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1252/2509] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:20:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:20:06.810875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e933e8-9d30-4bde-ad02-d1b1cb33b5ff None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:06.811439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e933e8-9d30-4bde-ad02-d1b1cb33b5ff None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:06.811907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e933e8-9d30-4bde-ad02-d1b1cb33b5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:06.832804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c3e933e8-9d30-4bde-ad02-d1b1cb33b5ff None None] MFA Rules not processed for user `98649aa38f764af0b41ff5594b94a989`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:06.834312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c3e933e8-9d30-4bde-ad02-d1b1cb33b5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:06.842636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1258/2510] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:20:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:06.855288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba50835-0835-49f1-be14-f165cca6838e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:06.857211 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4ba50835-0835-49f1-be14-f165cca6838e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:06.872523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba50835-0835-49f1-be14-f165cca6838e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:06.876842 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ada8c-6c72-414f-8f2f-a15696c2f562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:06.877767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ada8c-6c72-414f-8f2f-a15696c2f562 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:06.878246 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ada8c-6c72-414f-8f2f-a15696c2f562 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:06.878720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ada8c-6c72-414f-8f2f-a15696c2f562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:06.888853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ada8c-6c72-414f-8f2f-a15696c2f562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:06.891173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ada8c-6c72-414f-8f2f-a15696c2f562 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:06.905734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1253/2511] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:06.910792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e933e8-9d30-4bde-ad02-d1b1cb33b5ff None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:06.920893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e933e8-9d30-4bde-ad02-d1b1cb33b5ff None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:06.924999 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2ebb6-0cb2-4286-b79c-598a06beffdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:06.925740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2ebb6-0cb2-4286-b79c-598a06beffdf service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:06.926117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2ebb6-0cb2-4286-b79c-598a06beffdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:06.926470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2ebb6-0cb2-4286-b79c-598a06beffdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:06.928522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-49e2ebb6-0cb2-4286-b79c-598a06beffdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:07.000106 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078af4a4-6942-4dab-b2a1-8f9f199574f0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:07.000393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078af4a4-6942-4dab-b2a1-8f9f199574f0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:07.000872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078af4a4-6942-4dab-b2a1-8f9f199574f0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:07.002641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1254/2512] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 04 09:20:07.008027 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2ebb6-0cb2-4286-b79c-598a06beffdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:07.008737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569636fe-4b68-46c5-ac27-4bec75f20c50 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:07.009020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569636fe-4b68-46c5-ac27-4bec75f20c50 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:07.009357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2ebb6-0cb2-4286-b79c-598a06beffdf service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:07.009502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569636fe-4b68-46c5-ac27-4bec75f20c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:07.022340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1259/2513] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:07.053192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-569636fe-4b68-46c5-ac27-4bec75f20c50 None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:07.054550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-569636fe-4b68-46c5-ac27-4bec75f20c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:07.064427 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1255/2514] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:20:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:07.104311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2ebb6-0cb2-4286-b79c-598a06beffdf service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:07.114359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2ebb6-0cb2-4286-b79c-598a06beffdf service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:07.123505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa601f-b285-4d37-80fb-675accd17db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:07.125301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa601f-b285-4d37-80fb-675accd17db7 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:07.125680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa601f-b285-4d37-80fb-675accd17db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:07.126088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa601f-b285-4d37-80fb-675accd17db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:07.140736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa601f-b285-4d37-80fb-675accd17db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:07.142333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa601f-b285-4d37-80fb-675accd17db7 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:07.160621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1260/2515] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:07.288282 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569636fe-4b68-46c5-ac27-4bec75f20c50 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:07.290135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-569636fe-4b68-46c5-ac27-4bec75f20c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:07.374686 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569636fe-4b68-46c5-ac27-4bec75f20c50 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:07.379781 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162b91f-07fa-47bd-a0c7-905f5f9a39d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:07.380600 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162b91f-07fa-47bd-a0c7-905f5f9a39d5 service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:07.380883 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162b91f-07fa-47bd-a0c7-905f5f9a39d5 service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:07.381158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162b91f-07fa-47bd-a0c7-905f5f9a39d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:07.383248 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1162b91f-07fa-47bd-a0c7-905f5f9a39d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:07.395260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162b91f-07fa-47bd-a0c7-905f5f9a39d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:07.396868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162b91f-07fa-47bd-a0c7-905f5f9a39d5 service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:07.407271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1256/2516] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:10.985316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa601f-b285-4d37-80fb-675accd17db7 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:10.986781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-28fa601f-b285-4d37-80fb-675accd17db7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:11.002688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa601f-b285-4d37-80fb-675accd17db7 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.008004 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51603d5b-a76e-4238-b031-6de2d37f818b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.008895 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51603d5b-a76e-4238-b031-6de2d37f818b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.009181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51603d5b-a76e-4238-b031-6de2d37f818b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.009462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51603d5b-a76e-4238-b031-6de2d37f818b admin admin] PATH_INFO: `/v3/users/1e916bf18efe421895c8d7b28fcd55d5` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.010771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51603d5b-a76e-4238-b031-6de2d37f818b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e916bf18efe421895c8d7b28fcd55d5)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.012807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51603d5b-a76e-4238-b031-6de2d37f818b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.060861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-51603d5b-a76e-4238-b031-6de2d37f818b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e916bf18efe421895c8d7b28fcd55d5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.061559 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-51603d5b-a76e-4238-b031-6de2d37f818b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e916bf18efe421895c8d7b28fcd55d5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.070225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-51603d5b-a76e-4238-b031-6de2d37f818b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e916bf18efe421895c8d7b28fcd55d5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.076968 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-51603d5b-a76e-4238-b031-6de2d37f818b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e916bf18efe421895c8d7b28fcd55d5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.077611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-51603d5b-a76e-4238-b031-6de2d37f818b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e916bf18efe421895c8d7b28fcd55d5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.094655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1261/2517] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:10 2022] DELETE /identity/v3/users/1e916bf18efe421895c8d7b28fcd55d5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:11.279431 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162b91f-07fa-47bd-a0c7-905f5f9a39d5 service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:11.292188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162b91f-07fa-47bd-a0c7-905f5f9a39d5 service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.352108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.353396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.353976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.354402 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] PATH_INFO: `/v3/projects/48690ebc49824572a08f53597eeee661` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.356689 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] RBAC: Authorizing `identity:delete_project(project_id=48690ebc49824572a08f53597eeee661)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.359615 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.406951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:11.410195 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48690ebc49824572a08f53597eeee661', 'request_id': 'req-2af6d040-dcd3-45b3-ae3c-ac5db466665e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.419846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48690ebc49824572a08f53597eeee661', 'request_id': 'req-2af6d040-dcd3-45b3-ae3c-ac5db466665e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.426232 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48690ebc49824572a08f53597eeee661', 'request_id': 'req-2af6d040-dcd3-45b3-ae3c-ac5db466665e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.436754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] The token cache is being invalidate because project 48690ebc49824572a08f53597eeee661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:11.437037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.437716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2af6d040-dcd3-45b3-ae3c-ac5db466665e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:11.439301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1257/2518] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:11 2022] DELETE /identity/v3/projects/48690ebc49824572a08f53597eeee661 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:11.445443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4fec7-3488-4d71-8252-c2aa3d2a3677 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.445443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4fec7-3488-4d71-8252-c2aa3d2a3677 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.445443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4fec7-3488-4d71-8252-c2aa3d2a3677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.464175 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-5fa4fec7-3488-4d71-8252-c2aa3d2a3677 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:11.465386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-5fa4fec7-3488-4d71-8252-c2aa3d2a3677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:11.502881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:11.535025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1262/2519] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:20:11.542897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622bc96-400d-4183-98b4-6341940cc699 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.543184 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622bc96-400d-4183-98b4-6341940cc699 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.543486 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622bc96-400d-4183-98b4-6341940cc699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.564209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2622bc96-400d-4183-98b4-6341940cc699 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:11.565440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2622bc96-400d-4183-98b4-6341940cc699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:11.575388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1258/2520] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:11.579116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4fec7-3488-4d71-8252-c2aa3d2a3677 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:11.589359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4fec7-3488-4d71-8252-c2aa3d2a3677 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.594348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2a0bd-df34-411e-be7c-e6c7bba0d20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.595001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a0bd-df34-411e-be7c-e6c7bba0d20b admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.595278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a0bd-df34-411e-be7c-e6c7bba0d20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.595557 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a0bd-df34-411e-be7c-e6c7bba0d20b admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.596326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2a0bd-df34-411e-be7c-e6c7bba0d20b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:20:11.596636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2a0bd-df34-411e-be7c-e6c7bba0d20b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.598278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2a0bd-df34-411e-be7c-e6c7bba0d20b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.613524 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1263/2521] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:20:11.618153 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622bc96-400d-4183-98b4-6341940cc699 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:11.627984 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622bc96-400d-4183-98b4-6341940cc699 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.633102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2e860-8676-43f7-bbe1-47e4335debd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.633760 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2e860-8676-43f7-bbe1-47e4335debd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.634082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2e860-8676-43f7-bbe1-47e4335debd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.634354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2e860-8676-43f7-bbe1-47e4335debd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.635042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2e860-8676-43f7-bbe1-47e4335debd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.636941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2e860-8676-43f7-bbe1-47e4335debd3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.665552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1259/2522] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:20:11 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:20:11.670262 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2a0bd-df34-411e-be7c-e6c7bba0d20b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:11.681429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2a0bd-df34-411e-be7c-e6c7bba0d20b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.745867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4b531-0582-4ced-b64e-96ae78e1b3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.746590 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4b531-0582-4ced-b64e-96ae78e1b3cc admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.746880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4b531-0582-4ced-b64e-96ae78e1b3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.747157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4b531-0582-4ced-b64e-96ae78e1b3cc admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.747894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4b531-0582-4ced-b64e-96ae78e1b3cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.749803 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4b531-0582-4ced-b64e-96ae78e1b3cc admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.771695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1264/2523] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:20:11 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:20:11.775925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2e860-8676-43f7-bbe1-47e4335debd3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:11.786765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2e860-8676-43f7-bbe1-47e4335debd3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.791514 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43127fc-75d5-4ef4-befa-a7e2c9fa65da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.792174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43127fc-75d5-4ef4-befa-a7e2c9fa65da admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.792459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43127fc-75d5-4ef4-befa-a7e2c9fa65da admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.792737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43127fc-75d5-4ef4-befa-a7e2c9fa65da admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.793529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43127fc-75d5-4ef4-befa-a7e2c9fa65da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.795187 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43127fc-75d5-4ef4-befa-a7e2c9fa65da admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.806220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1260/2524] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:20:11.810230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4b531-0582-4ced-b64e-96ae78e1b3cc admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:11.821260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4b531-0582-4ced-b64e-96ae78e1b3cc admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.827239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396096e-dc23-4261-bb48-a6af61d26ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.827953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396096e-dc23-4261-bb48-a6af61d26ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.828230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396096e-dc23-4261-bb48-a6af61d26ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.828503 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396096e-dc23-4261-bb48-a6af61d26ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.829234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396096e-dc23-4261-bb48-a6af61d26ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.830908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396096e-dc23-4261-bb48-a6af61d26ec0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.841644 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1265/2525] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:20:11.845856 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43127fc-75d5-4ef4-befa-a7e2c9fa65da admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:11.856671 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43127fc-75d5-4ef4-befa-a7e2c9fa65da admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:11.862668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51abad-5fd5-4bbc-afb4-4e1ac98978b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:11.863553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51abad-5fd5-4bbc-afb4-4e1ac98978b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.863828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51abad-5fd5-4bbc-afb4-4e1ac98978b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.864100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51abad-5fd5-4bbc-afb4-4e1ac98978b9 admin admin] PATH_INFO: `/v3/projects/4677b4c393ff41a09c922467365e06f0/users/bcad5b2163e647f386b482731e1de798/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.881696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51abad-5fd5-4bbc-afb4-4e1ac98978b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4677b4c393ff41a09c922467365e06f0, user_id=bcad5b2163e647f386b482731e1de798, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:11.884529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51abad-5fd5-4bbc-afb4-4e1ac98978b9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:11.900853 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1261/2526] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:20:11 2022] PUT /identity/v3/projects/4677b4c393ff41a09c922467365e06f0/users/bcad5b2163e647f386b482731e1de798/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:11.907807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751860da-de67-4199-9177-8acc1035e58c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:11.908101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751860da-de67-4199-9177-8acc1035e58c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:11.908378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751860da-de67-4199-9177-8acc1035e58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:11.928101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-751860da-de67-4199-9177-8acc1035e58c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:11.929469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-751860da-de67-4199-9177-8acc1035e58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:11.995149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1266/2527] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:12.005127 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51abad-5fd5-4bbc-afb4-4e1ac98978b9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:12.006917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0f51abad-5fd5-4bbc-afb4-4e1ac98978b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:12.065472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51abad-5fd5-4bbc-afb4-4e1ac98978b9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:12.071043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d327e-6577-4f74-b0fe-eef1e8d5e3fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:12.071704 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d327e-6577-4f74-b0fe-eef1e8d5e3fd service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:12.071986 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d327e-6577-4f74-b0fe-eef1e8d5e3fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:12.072257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d327e-6577-4f74-b0fe-eef1e8d5e3fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:12.086390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d327e-6577-4f74-b0fe-eef1e8d5e3fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:12.087847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d327e-6577-4f74-b0fe-eef1e8d5e3fd service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:12.097754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1262/2528] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:16.123459 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00627bb8-c4a4-498d-b9ad-64226194a935 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:16.124683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00627bb8-c4a4-498d-b9ad-64226194a935 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:16.124987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00627bb8-c4a4-498d-b9ad-64226194a935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:16.154547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-00627bb8-c4a4-498d-b9ad-64226194a935 None None] MFA Rules not processed for user `bcad5b2163e647f386b482731e1de798`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:16.156924 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-00627bb8-c4a4-498d-b9ad-64226194a935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:16.215153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:16.254853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1267/2529] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:20:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:16.270708 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d327e-6577-4f74-b0fe-eef1e8d5e3fd service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:16.272371 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-eb9d327e-6577-4f74-b0fe-eef1e8d5e3fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:16.355043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d327e-6577-4f74-b0fe-eef1e8d5e3fd service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:16.361965 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e288a3-b724-4e83-92d2-b9941f6c9ca6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:16.362521 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e288a3-b724-4e83-92d2-b9941f6c9ca6 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:16.362801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e288a3-b724-4e83-92d2-b9941f6c9ca6 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:16.363085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e288a3-b724-4e83-92d2-b9941f6c9ca6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:16.373397 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e288a3-b724-4e83-92d2-b9941f6c9ca6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:16.375124 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e288a3-b724-4e83-92d2-b9941f6c9ca6 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:16.379200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:16.386386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1263/2530] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:17.808489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a810f2b-1932-4df3-a747-7a776819a7f4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:17.809015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a810f2b-1932-4df3-a747-7a776819a7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:17.809121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a810f2b-1932-4df3-a747-7a776819a7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:17.827076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1a810f2b-1932-4df3-a747-7a776819a7f4 None None] MFA Rules not processed for user `bcad5b2163e647f386b482731e1de798`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:17.828346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1a810f2b-1932-4df3-a747-7a776819a7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:17.836384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1268/2531] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:20:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:18.165513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e288a3-b724-4e83-92d2-b9941f6c9ca6 service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:18.175882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e288a3-b724-4e83-92d2-b9941f6c9ca6 service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:18.180086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a8de7-504b-4bca-a4e7-7374d59c6c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:18.180749 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a8de7-504b-4bca-a4e7-7374d59c6c0f service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:18.181037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a8de7-504b-4bca-a4e7-7374d59c6c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:18.181312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a8de7-504b-4bca-a4e7-7374d59c6c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:18.183212 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-920a8de7-504b-4bca-a4e7-7374d59c6c0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:18.193479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a8de7-504b-4bca-a4e7-7374d59c6c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:18.194958 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a8de7-504b-4bca-a4e7-7374d59c6c0f service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:18.203670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1264/2532] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:19.718028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a810f2b-1932-4df3-a747-7a776819a7f4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:19.720147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1a810f2b-1932-4df3-a747-7a776819a7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:19.739789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a810f2b-1932-4df3-a747-7a776819a7f4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:19.747581 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a624e8-b01e-43b9-a012-e316d920b19f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:19.748256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a624e8-b01e-43b9-a012-e316d920b19f service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:19.748540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a624e8-b01e-43b9-a012-e316d920b19f service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:19.748816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a624e8-b01e-43b9-a012-e316d920b19f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:19.750717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-34a624e8-b01e-43b9-a012-e316d920b19f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:19.761322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a624e8-b01e-43b9-a012-e316d920b19f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:19.763253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a624e8-b01e-43b9-a012-e316d920b19f service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:19.774106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1269/2533] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:22.214916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a8de7-504b-4bca-a4e7-7374d59c6c0f service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:22.235219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a8de7-504b-4bca-a4e7-7374d59c6c0f service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:22.243041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be875536-eb09-4f27-90a2-7f769f66be32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:22.243735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be875536-eb09-4f27-90a2-7f769f66be32 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:22.244087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be875536-eb09-4f27-90a2-7f769f66be32 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:22.244454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be875536-eb09-4f27-90a2-7f769f66be32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:22.246420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-be875536-eb09-4f27-90a2-7f769f66be32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:22.262838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be875536-eb09-4f27-90a2-7f769f66be32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:22.264546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be875536-eb09-4f27-90a2-7f769f66be32 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:22.278382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1265/2534] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:20:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:20:29.769277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=23 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:32.273050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=13 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:43.782003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4542-c7da-41f3-a33b-9fe774f763d3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:43.782501 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4542-c7da-41f3-a33b-9fe774f763d3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:43.782614 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4542-c7da-41f3-a33b-9fe774f763d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:43.801731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-bb8c4542-c7da-41f3-a33b-9fe774f763d3 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:43.802954 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-bb8c4542-c7da-41f3-a33b-9fe774f763d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:43.858970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1270/2535] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 4 09:20:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:43.900643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be875536-eb09-4f27-90a2-7f769f66be32 service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:43.902148 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-be875536-eb09-4f27-90a2-7f769f66be32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:43.977645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be875536-eb09-4f27-90a2-7f769f66be32 service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:43.983970 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952befbc-51d1-4b44-8580-95869ced4787 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:43.984667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952befbc-51d1-4b44-8580-95869ced4787 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:43.984953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952befbc-51d1-4b44-8580-95869ced4787 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:43.985329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952befbc-51d1-4b44-8580-95869ced4787 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:43.994910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952befbc-51d1-4b44-8580-95869ced4787 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:43.997377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952befbc-51d1-4b44-8580-95869ced4787 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:44.006803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1266/2536] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:20:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:20:49.657105 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eef7dc-3ecf-4231-a10f-4c03e78d652c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:49.658691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eef7dc-3ecf-4231-a10f-4c03e78d652c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:49.658918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eef7dc-3ecf-4231-a10f-4c03e78d652c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:49.682751 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e5eef7dc-3ecf-4231-a10f-4c03e78d652c None None] MFA Rules not processed for user `f884304cbb764a8ab4927c3ba8db892d`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:20:49.683993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e5eef7dc-3ecf-4231-a10f-4c03e78d652c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:49.764517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1271/2537] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:20:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:20:51.955133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952befbc-51d1-4b44-8580-95869ced4787 service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:51.957153 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-952befbc-51d1-4b44-8580-95869ced4787 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:51.996045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952befbc-51d1-4b44-8580-95869ced4787 service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:52.002202 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:52.003350 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:52.003702 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:52.004129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] PATH_INFO: `/v3/users/a9dfef8fef8d45d99636579271c71d51` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:52.005538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9dfef8fef8d45d99636579271c71d51)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:52.007787 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:52.074635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9dfef8fef8d45d99636579271c71d51'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.075237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9dfef8fef8d45d99636579271c71d51'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.075994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9dfef8fef8d45d99636579271c71d51'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.086174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9dfef8fef8d45d99636579271c71d51'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.096664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9dfef8fef8d45d99636579271c71d51'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.121030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1267/2538] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:51 2022] DELETE /identity/v3/users/a9dfef8fef8d45d99636579271c71d51 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:52.366053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eef7dc-3ecf-4231-a10f-4c03e78d652c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:52.383561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eef7dc-3ecf-4231-a10f-4c03e78d652c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:52.454058 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:52.454936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:52.455215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:52.455520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] PATH_INFO: `/v3/projects/8a4eaafcb5814b92a0c740931fcf422d` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:52.456887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a4eaafcb5814b92a0c740931fcf422d)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:52.459326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:52.510816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a4eaafcb5814b92a0c740931fcf422d', 'request_id': 'req-fb8f3f44-929c-497b-9e17-1f8428802233'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.514226 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:52.528541 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a4eaafcb5814b92a0c740931fcf422d', 'request_id': 'req-fb8f3f44-929c-497b-9e17-1f8428802233'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.535975 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a4eaafcb5814b92a0c740931fcf422d', 'request_id': 'req-fb8f3f44-929c-497b-9e17-1f8428802233'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.556970 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] The token cache is being invalidate because project 8a4eaafcb5814b92a0c740931fcf422d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:52.557246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.558003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.559656 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1272/2539] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:52 2022] DELETE /identity/v3/projects/8a4eaafcb5814b92a0c740931fcf422d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:52.567293 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:52.569452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:52.585599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5568de4-1cfa-4636-925c-42aab21a8569 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:52.703263 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:52.704483 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:52.704931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:52.706043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] PATH_INFO: `/v3/users/7ddbac08c11747da9b343135b4eb438c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:52.714577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ddbac08c11747da9b343135b4eb438c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:52.716669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:52.720346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:20:52.794343 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ddbac08c11747da9b343135b4eb438c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.795056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ddbac08c11747da9b343135b4eb438c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.795664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ddbac08c11747da9b343135b4eb438c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.805795 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ddbac08c11747da9b343135b4eb438c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.814421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ddbac08c11747da9b343135b4eb438c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:52.838152 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1268/2540] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:20:52 2022] DELETE /identity/v3/users/7ddbac08c11747da9b343135b4eb438c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:20:53.028934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:20:53.030551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:20:53.044944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f3f44-929c-497b-9e17-1f8428802233 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:20:53.122415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:20:53.123682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:20:53.124035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:20:53.124791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] PATH_INFO: `/v3/projects/a110bfdb567344a592e26dca6f9a7d12` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:20:53.127036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a110bfdb567344a592e26dca6f9a7d12)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:20:53.129096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:20:53.182081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a110bfdb567344a592e26dca6f9a7d12', 'request_id': 'req-f88a0dbe-c713-4402-940c-3edde23a3d7a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:53.191187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a110bfdb567344a592e26dca6f9a7d12', 'request_id': 'req-f88a0dbe-c713-4402-940c-3edde23a3d7a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:53.198164 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a110bfdb567344a592e26dca6f9a7d12', 'request_id': 'req-f88a0dbe-c713-4402-940c-3edde23a3d7a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:53.209589 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] The token cache is being invalidate because project a110bfdb567344a592e26dca6f9a7d12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:20:53.209909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:53.210608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:20:53.212230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1273/2541] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:20:53 2022] DELETE /identity/v3/projects/a110bfdb567344a592e26dca6f9a7d12 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:02.834560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:03.206270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:03.328355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:03.330109 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:03.419506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a98a06-01a6-4c9a-a632-2a2e382b5015 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:03.489252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440fc81-1ac9-4572-87e7-b3ecb53e5fa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:03.489969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440fc81-1ac9-4572-87e7-b3ecb53e5fa2 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:03.490260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440fc81-1ac9-4572-87e7-b3ecb53e5fa2 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:03.490541 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440fc81-1ac9-4572-87e7-b3ecb53e5fa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:03.492436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b440fc81-1ac9-4572-87e7-b3ecb53e5fa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:03.504584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440fc81-1ac9-4572-87e7-b3ecb53e5fa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:03.506405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440fc81-1ac9-4572-87e7-b3ecb53e5fa2 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:03.517530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1269/2542] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:13.127225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:13.128770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:13.190046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a0dbe-c713-4402-940c-3edde23a3d7a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:13.194048 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419d314-722a-4da8-b1d4-3778d0938f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:13.194690 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419d314-722a-4da8-b1d4-3778d0938f03 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:13.194977 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419d314-722a-4da8-b1d4-3778d0938f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:13.195245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419d314-722a-4da8-b1d4-3778d0938f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:13.197042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2419d314-722a-4da8-b1d4-3778d0938f03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:13.267642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419d314-722a-4da8-b1d4-3778d0938f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:13.269087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419d314-722a-4da8-b1d4-3778d0938f03 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:13.280040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1274/2543] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:13.512654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=62 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:16.536531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440fc81-1ac9-4572-87e7-b3ecb53e5fa2 service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:16.538308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b440fc81-1ac9-4572-87e7-b3ecb53e5fa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:16.553665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440fc81-1ac9-4572-87e7-b3ecb53e5fa2 service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:16.559519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57c66f-6d62-4dfd-b4e1-2d4abe83c129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:16.560579 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57c66f-6d62-4dfd-b4e1-2d4abe83c129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:16.560917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57c66f-6d62-4dfd-b4e1-2d4abe83c129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:16.561250 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57c66f-6d62-4dfd-b4e1-2d4abe83c129 admin admin] PATH_INFO: `/v3/users/43bcd8fb25604011bfd1df823ea4ddad` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:16.562643 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57c66f-6d62-4dfd-b4e1-2d4abe83c129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43bcd8fb25604011bfd1df823ea4ddad)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:16.564764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57c66f-6d62-4dfd-b4e1-2d4abe83c129 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:16.613689 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2d57c66f-6d62-4dfd-b4e1-2d4abe83c129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43bcd8fb25604011bfd1df823ea4ddad'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:16.614401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2d57c66f-6d62-4dfd-b4e1-2d4abe83c129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43bcd8fb25604011bfd1df823ea4ddad'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:16.615038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2d57c66f-6d62-4dfd-b4e1-2d4abe83c129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43bcd8fb25604011bfd1df823ea4ddad'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:16.623362 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2d57c66f-6d62-4dfd-b4e1-2d4abe83c129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43bcd8fb25604011bfd1df823ea4ddad'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:16.631059 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2d57c66f-6d62-4dfd-b4e1-2d4abe83c129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43bcd8fb25604011bfd1df823ea4ddad'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:16.655751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1270/2544] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:16 2022] DELETE /identity/v3/users/43bcd8fb25604011bfd1df823ea4ddad => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:16.836290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419d314-722a-4da8-b1d4-3778d0938f03 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:16.847716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419d314-722a-4da8-b1d4-3778d0938f03 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:16.906993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:16.907795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:16.908075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:16.908353 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] PATH_INFO: `/v3/projects/8cfd9d65f66641b2bda6bbdcf908ca3c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:16.909636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cfd9d65f66641b2bda6bbdcf908ca3c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:16.911649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:16.963685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cfd9d65f66641b2bda6bbdcf908ca3c', 'request_id': 'req-007f599a-6a20-4d4e-b774-ce6d8548a640'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:16.966820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:16.973775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cfd9d65f66641b2bda6bbdcf908ca3c', 'request_id': 'req-007f599a-6a20-4d4e-b774-ce6d8548a640'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:16.981858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cfd9d65f66641b2bda6bbdcf908ca3c', 'request_id': 'req-007f599a-6a20-4d4e-b774-ce6d8548a640'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:16.993272 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] The token cache is being invalidate because project 8cfd9d65f66641b2bda6bbdcf908ca3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:16.993563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:16.994210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-007f599a-6a20-4d4e-b774-ce6d8548a640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:16.995773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1275/2545] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:16 2022] DELETE /identity/v3/projects/8cfd9d65f66641b2bda6bbdcf908ca3c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:21:17.001378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22863ec6-59ef-4d5b-97c3-5c991e307ddd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:17.001803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22863ec6-59ef-4d5b-97c3-5c991e307ddd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:17.002166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22863ec6-59ef-4d5b-97c3-5c991e307ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:17.023544 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-22863ec6-59ef-4d5b-97c3-5c991e307ddd None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:17.024770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-22863ec6-59ef-4d5b-97c3-5c991e307ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:17.098708 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1271/2546] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:17.107479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87855a86-115e-47f7-9cfe-22ad24a3e99c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:17.107760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87855a86-115e-47f7-9cfe-22ad24a3e99c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:17.108068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87855a86-115e-47f7-9cfe-22ad24a3e99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:17.128318 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-87855a86-115e-47f7-9cfe-22ad24a3e99c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:17.129603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-87855a86-115e-47f7-9cfe-22ad24a3e99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:17.140695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1276/2547] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:17.150421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22863ec6-59ef-4d5b-97c3-5c991e307ddd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:17.161770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22863ec6-59ef-4d5b-97c3-5c991e307ddd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:17.167651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0daf498-79f3-45a7-bfb8-56f7ada113b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:17.168450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daf498-79f3-45a7-bfb8-56f7ada113b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:17.168731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daf498-79f3-45a7-bfb8-56f7ada113b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:17.169010 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daf498-79f3-45a7-bfb8-56f7ada113b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:17.169753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0daf498-79f3-45a7-bfb8-56f7ada113b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:21:17.170105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0daf498-79f3-45a7-bfb8-56f7ada113b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:17.171711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0daf498-79f3-45a7-bfb8-56f7ada113b5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:17.187845 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1272/2548] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:21:17.189443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87855a86-115e-47f7-9cfe-22ad24a3e99c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:17.198802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87855a86-115e-47f7-9cfe-22ad24a3e99c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:17.203937 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee680ce0-ea5b-4636-a27e-a7e4bbabc996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:17.204602 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee680ce0-ea5b-4636-a27e-a7e4bbabc996 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:17.204883 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee680ce0-ea5b-4636-a27e-a7e4bbabc996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:17.205155 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee680ce0-ea5b-4636-a27e-a7e4bbabc996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:17.205847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee680ce0-ea5b-4636-a27e-a7e4bbabc996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:17.207791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee680ce0-ea5b-4636-a27e-a7e4bbabc996 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:17.247510 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1277/2549] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:17 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:17.251827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0daf498-79f3-45a7-bfb8-56f7ada113b5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:17.262880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0daf498-79f3-45a7-bfb8-56f7ada113b5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:17.330070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82d806-fad5-4c68-8f5b-e036b7b48d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:17.330775 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82d806-fad5-4c68-8f5b-e036b7b48d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:17.331217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82d806-fad5-4c68-8f5b-e036b7b48d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:17.331630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82d806-fad5-4c68-8f5b-e036b7b48d16 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:17.332320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82d806-fad5-4c68-8f5b-e036b7b48d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:17.334287 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82d806-fad5-4c68-8f5b-e036b7b48d16 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:17.341103 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:17.357286 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1273/2550] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:17 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:21:17.361459 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee680ce0-ea5b-4636-a27e-a7e4bbabc996 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:17.373627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee680ce0-ea5b-4636-a27e-a7e4bbabc996 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:17.378411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c698a-7199-4ba6-9c22-b3a45922df19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:17.379075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c698a-7199-4ba6-9c22-b3a45922df19 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:17.379359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c698a-7199-4ba6-9c22-b3a45922df19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:17.379638 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c698a-7199-4ba6-9c22-b3a45922df19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:17.380367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c698a-7199-4ba6-9c22-b3a45922df19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:17.382215 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c698a-7199-4ba6-9c22-b3a45922df19 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:17.393013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1278/2551] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:17.397146 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82d806-fad5-4c68-8f5b-e036b7b48d16 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:17.407396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82d806-fad5-4c68-8f5b-e036b7b48d16 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:17.412676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab73ec6-33d2-4dbc-b705-a6d9808e6a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:17.413339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab73ec6-33d2-4dbc-b705-a6d9808e6a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:17.413614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab73ec6-33d2-4dbc-b705-a6d9808e6a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:17.413938 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab73ec6-33d2-4dbc-b705-a6d9808e6a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:17.414774 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab73ec6-33d2-4dbc-b705-a6d9808e6a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:17.416401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab73ec6-33d2-4dbc-b705-a6d9808e6a9a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:17.430333 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1274/2552] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:17.432688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c698a-7199-4ba6-9c22-b3a45922df19 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:17.443361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c698a-7199-4ba6-9c22-b3a45922df19 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:17.448686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5b57a-46ee-4cca-abca-5969046a2c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:17.450179 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5b57a-46ee-4cca-abca-5969046a2c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:17.450466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5b57a-46ee-4cca-abca-5969046a2c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:17.450840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5b57a-46ee-4cca-abca-5969046a2c3f admin admin] PATH_INFO: `/v3/projects/d855f761a66d4fc08d740b382f88368c/users/adc52d3dd63f493c99f6145d4e6bfb4e/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:17.467746 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5b57a-46ee-4cca-abca-5969046a2c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d855f761a66d4fc08d740b382f88368c, user_id=adc52d3dd63f493c99f6145d4e6bfb4e, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:17.470867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5b57a-46ee-4cca-abca-5969046a2c3f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:17.486177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1279/2553] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:17 2022] PUT /identity/v3/projects/d855f761a66d4fc08d740b382f88368c/users/adc52d3dd63f493c99f6145d4e6bfb4e/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:17.496757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88503b-9261-4dce-8a58-f68ac39b2719 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:17.497074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88503b-9261-4dce-8a58-f68ac39b2719 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:17.497354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88503b-9261-4dce-8a58-f68ac39b2719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:17.525486 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8b88503b-9261-4dce-8a58-f68ac39b2719 None None] MFA Rules not processed for user `adc52d3dd63f493c99f6145d4e6bfb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:17.526815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8b88503b-9261-4dce-8a58-f68ac39b2719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:17.593779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1275/2554] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:17.604750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5b57a-46ee-4cca-abca-5969046a2c3f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:17.606543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-96e5b57a-46ee-4cca-abca-5969046a2c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:17.666603 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5b57a-46ee-4cca-abca-5969046a2c3f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:17.670768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fdcf1-037a-4fb3-9fdb-01a71494dca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:17.671416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fdcf1-037a-4fb3-9fdb-01a71494dca2 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:17.671700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fdcf1-037a-4fb3-9fdb-01a71494dca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:17.671976 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fdcf1-037a-4fb3-9fdb-01a71494dca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:17.680463 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fdcf1-037a-4fb3-9fdb-01a71494dca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:17.681921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fdcf1-037a-4fb3-9fdb-01a71494dca2 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:17.690892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1280/2555] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:17 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:26.099897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88503b-9261-4dce-8a58-f68ac39b2719 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:26.173332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88503b-9261-4dce-8a58-f68ac39b2719 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:26.178413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:26.179283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:26.179564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:26.179835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] PATH_INFO: `/v3/users/adc52d3dd63f493c99f6145d4e6bfb4e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:26.181019 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adc52d3dd63f493c99f6145d4e6bfb4e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:26.182988 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:26.238495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adc52d3dd63f493c99f6145d4e6bfb4e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:26.239198 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adc52d3dd63f493c99f6145d4e6bfb4e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:26.239832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adc52d3dd63f493c99f6145d4e6bfb4e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:26.248762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adc52d3dd63f493c99f6145d4e6bfb4e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:26.256494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adc52d3dd63f493c99f6145d4e6bfb4e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:26.262020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:26.276724 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1276/2556] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:26 2022] DELETE /identity/v3/users/adc52d3dd63f493c99f6145d4e6bfb4e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:26.281526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f15650-2720-4761-a335-1e2924569aa3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:26.282050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f15650-2720-4761-a335-1e2924569aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:26.282149 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f15650-2720-4761-a335-1e2924569aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:26.300131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b8f15650-2720-4761-a335-1e2924569aa3 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:26.301329 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b8f15650-2720-4761-a335-1e2924569aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:26.365338 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1281/2557] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:26.376015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:26.378004 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:26.438882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e37e6-f622-48f2-a614-5a8c56accdb2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:26.442821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661da08-c4db-44da-b025-96272fc982f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:26.443470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661da08-c4db-44da-b025-96272fc982f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:26.443753 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661da08-c4db-44da-b025-96272fc982f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:26.444021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661da08-c4db-44da-b025-96272fc982f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:26.453654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661da08-c4db-44da-b025-96272fc982f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:26.455145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661da08-c4db-44da-b025-96272fc982f7 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:26.464770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1277/2558] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:26.747757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f15650-2720-4761-a335-1e2924569aa3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:26.749359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b8f15650-2720-4761-a335-1e2924569aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:26.761877 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f15650-2720-4761-a335-1e2924569aa3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:26.768822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:26.769810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:26.770191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:26.770410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] PATH_INFO: `/v3/projects/d855f761a66d4fc08d740b382f88368c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:26.771706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d855f761a66d4fc08d740b382f88368c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:26.774595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:26.782101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:26.830749 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd855f761a66d4fc08d740b382f88368c', 'request_id': 'req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:26.839030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd855f761a66d4fc08d740b382f88368c', 'request_id': 'req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:26.845801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd855f761a66d4fc08d740b382f88368c', 'request_id': 'req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:26.856762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] The token cache is being invalidate because project d855f761a66d4fc08d740b382f88368c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:26.857067 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:26.857706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-e3e2bd53-cc57-4587-8c3a-6bb77cb597b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:26.859355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1282/2559] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:26 2022] DELETE /identity/v3/projects/d855f761a66d4fc08d740b382f88368c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:26.864279 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce686148-251d-407a-973f-82fffaf06081 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:26.864577 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce686148-251d-407a-973f-82fffaf06081 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:26.864858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce686148-251d-407a-973f-82fffaf06081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:26.883612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ce686148-251d-407a-973f-82fffaf06081 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:26.884754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ce686148-251d-407a-973f-82fffaf06081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:26.947262 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1278/2560] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:26.955335 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe20e44-7523-44aa-aa8f-bb1925c4ca86 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:26.955634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe20e44-7523-44aa-aa8f-bb1925c4ca86 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:26.955902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe20e44-7523-44aa-aa8f-bb1925c4ca86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:26.974805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9fe20e44-7523-44aa-aa8f-bb1925c4ca86 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:26.976007 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9fe20e44-7523-44aa-aa8f-bb1925c4ca86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:26.984120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1283/2561] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:26.988593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce686148-251d-407a-973f-82fffaf06081 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:26.998654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce686148-251d-407a-973f-82fffaf06081 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.004619 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244f69c-c83a-4f63-ba4c-d6aba8f216fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.005330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244f69c-c83a-4f63-ba4c-d6aba8f216fd admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.005669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244f69c-c83a-4f63-ba4c-d6aba8f216fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.006013 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244f69c-c83a-4f63-ba4c-d6aba8f216fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.006976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244f69c-c83a-4f63-ba4c-d6aba8f216fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:21:27.007329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244f69c-c83a-4f63-ba4c-d6aba8f216fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.009135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244f69c-c83a-4f63-ba4c-d6aba8f216fd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.021737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1279/2562] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:21:27.025927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe20e44-7523-44aa-aa8f-bb1925c4ca86 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:27.035500 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe20e44-7523-44aa-aa8f-bb1925c4ca86 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.040860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669fb3d-5683-469c-9c17-5ed409afafca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.041528 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669fb3d-5683-469c-9c17-5ed409afafca admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.041808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669fb3d-5683-469c-9c17-5ed409afafca admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.042120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669fb3d-5683-469c-9c17-5ed409afafca admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.042812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669fb3d-5683-469c-9c17-5ed409afafca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.044708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669fb3d-5683-469c-9c17-5ed409afafca admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.073408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1284/2563] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:27 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:27.077488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244f69c-c83a-4f63-ba4c-d6aba8f216fd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:27.088706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244f69c-c83a-4f63-ba4c-d6aba8f216fd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.152018 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e247ab2-d273-45b8-972c-2ba86e89395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.152706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e247ab2-d273-45b8-972c-2ba86e89395d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.152945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e247ab2-d273-45b8-972c-2ba86e89395d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.153117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e247ab2-d273-45b8-972c-2ba86e89395d admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.153841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e247ab2-d273-45b8-972c-2ba86e89395d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.156356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e247ab2-d273-45b8-972c-2ba86e89395d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.178739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1280/2564] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:27 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:21:27.182988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669fb3d-5683-469c-9c17-5ed409afafca admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:27.193560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669fb3d-5683-469c-9c17-5ed409afafca admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.198606 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65455dc6-da03-4455-88e3-bec6cb1af401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.199270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455dc6-da03-4455-88e3-bec6cb1af401 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.199553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455dc6-da03-4455-88e3-bec6cb1af401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.199824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455dc6-da03-4455-88e3-bec6cb1af401 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.200591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65455dc6-da03-4455-88e3-bec6cb1af401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.202343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65455dc6-da03-4455-88e3-bec6cb1af401 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.213246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1285/2565] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:27.217098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e247ab2-d273-45b8-972c-2ba86e89395d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:27.227435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e247ab2-d273-45b8-972c-2ba86e89395d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.232621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08de4e-73f2-486d-996d-0f2f36e85aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.233285 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08de4e-73f2-486d-996d-0f2f36e85aef admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.233560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08de4e-73f2-486d-996d-0f2f36e85aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.233838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08de4e-73f2-486d-996d-0f2f36e85aef admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.234622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08de4e-73f2-486d-996d-0f2f36e85aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.236210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08de4e-73f2-486d-996d-0f2f36e85aef admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.237525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:27.249764 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1281/2566] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:27.253963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65455dc6-da03-4455-88e3-bec6cb1af401 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:27.269337 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65455dc6-da03-4455-88e3-bec6cb1af401 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.275175 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8f11e-70ca-49e6-a9a5-3554ffd866eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.276064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8f11e-70ca-49e6-a9a5-3554ffd866eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.276349 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8f11e-70ca-49e6-a9a5-3554ffd866eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.276627 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8f11e-70ca-49e6-a9a5-3554ffd866eb admin admin] PATH_INFO: `/v3/projects/0dc41cc947d3428ab91cad237fe4982e/users/e661c3e8dd154aecb0dfc6e46c60ab75/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.294216 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8f11e-70ca-49e6-a9a5-3554ffd866eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc41cc947d3428ab91cad237fe4982e, user_id=e661c3e8dd154aecb0dfc6e46c60ab75, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.297138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8f11e-70ca-49e6-a9a5-3554ffd866eb admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.313913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1286/2567] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:27 2022] PUT /identity/v3/projects/0dc41cc947d3428ab91cad237fe4982e/users/e661c3e8dd154aecb0dfc6e46c60ab75/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:27.332224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cdcf6-b47c-4ec1-a7af-05c633e470b0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.332520 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cdcf6-b47c-4ec1-a7af-05c633e470b0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.332797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cdcf6-b47c-4ec1-a7af-05c633e470b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.361506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e73cdcf6-b47c-4ec1-a7af-05c633e470b0 None None] MFA Rules not processed for user `e661c3e8dd154aecb0dfc6e46c60ab75`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:27.362790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e73cdcf6-b47c-4ec1-a7af-05c633e470b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:27.431900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1282/2568] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:27.437404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8f11e-70ca-49e6-a9a5-3554ffd866eb admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:27.502756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8f11e-70ca-49e6-a9a5-3554ffd866eb admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.508298 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab207df-9346-436e-8d03-242667133aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.508997 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab207df-9346-436e-8d03-242667133aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.509278 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab207df-9346-436e-8d03-242667133aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.509545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab207df-9346-436e-8d03-242667133aa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.510260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab207df-9346-436e-8d03-242667133aa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.512158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab207df-9346-436e-8d03-242667133aa9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.541671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1287/2569] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:27 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:27.547661 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cdcf6-b47c-4ec1-a7af-05c633e470b0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:27.564049 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cdcf6-b47c-4ec1-a7af-05c633e470b0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.632598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76852-013f-44f4-85d2-7076962121fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.633271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76852-013f-44f4-85d2-7076962121fc admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.633546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76852-013f-44f4-85d2-7076962121fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.633850 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76852-013f-44f4-85d2-7076962121fc admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.634641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a76852-013f-44f4-85d2-7076962121fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.636647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a76852-013f-44f4-85d2-7076962121fc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.658735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1283/2570] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:27 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:27.663002 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab207df-9346-436e-8d03-242667133aa9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:27.673638 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab207df-9346-436e-8d03-242667133aa9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.678956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9969956-af00-49c4-89e8-92beb23fbb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.679624 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9969956-af00-49c4-89e8-92beb23fbb41 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.679963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9969956-af00-49c4-89e8-92beb23fbb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.680181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9969956-af00-49c4-89e8-92beb23fbb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.680941 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9969956-af00-49c4-89e8-92beb23fbb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.682636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9969956-af00-49c4-89e8-92beb23fbb41 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.694295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1288/2571] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:27.698426 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76852-013f-44f4-85d2-7076962121fc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:27.709182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76852-013f-44f4-85d2-7076962121fc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.714933 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee27e6-d426-475a-abc2-f307ac9b19db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.715823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee27e6-d426-475a-abc2-f307ac9b19db admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.716093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee27e6-d426-475a-abc2-f307ac9b19db admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.716367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee27e6-d426-475a-abc2-f307ac9b19db admin admin] PATH_INFO: `/v3/projects/033a85a781b74c4089c90a60504fdf0c/users/b158289884b944119ab956aa7e31ed1f/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.731692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee27e6-d426-475a-abc2-f307ac9b19db admin admin] RBAC: Authorizing `identity:create_grant(project_id=033a85a781b74c4089c90a60504fdf0c, user_id=b158289884b944119ab956aa7e31ed1f, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.734396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee27e6-d426-475a-abc2-f307ac9b19db admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.749724 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1284/2572] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:27 2022] PUT /identity/v3/projects/033a85a781b74c4089c90a60504fdf0c/users/b158289884b944119ab956aa7e31ed1f/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:27.753959 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9969956-af00-49c4-89e8-92beb23fbb41 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:27.760247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:27.763986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9969956-af00-49c4-89e8-92beb23fbb41 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.823596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfeaa8d-ec13-4522-ba4e-25b5e7d041d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.824273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfeaa8d-ec13-4522-ba4e-25b5e7d041d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.824554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfeaa8d-ec13-4522-ba4e-25b5e7d041d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.824826 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfeaa8d-ec13-4522-ba4e-25b5e7d041d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.825554 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfeaa8d-ec13-4522-ba4e-25b5e7d041d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.827208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfeaa8d-ec13-4522-ba4e-25b5e7d041d1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.837521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1289/2573] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:27.841758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee27e6-d426-475a-abc2-f307ac9b19db admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:27.853065 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee27e6-d426-475a-abc2-f307ac9b19db admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:27.859079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847d91f-36a5-4ff6-a418-574d163a15bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:27.859974 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847d91f-36a5-4ff6-a418-574d163a15bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.860259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847d91f-36a5-4ff6-a418-574d163a15bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.860722 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847d91f-36a5-4ff6-a418-574d163a15bb admin admin] PATH_INFO: `/v3/projects/033a85a781b74c4089c90a60504fdf0c/users/b158289884b944119ab956aa7e31ed1f/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.862997 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847d91f-36a5-4ff6-a418-574d163a15bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=033a85a781b74c4089c90a60504fdf0c, user_id=b158289884b944119ab956aa7e31ed1f, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:27.865640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847d91f-36a5-4ff6-a418-574d163a15bb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:27.876921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1285/2574] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:27 2022] PUT /identity/v3/projects/033a85a781b74c4089c90a60504fdf0c/users/b158289884b944119ab956aa7e31ed1f/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:27.887280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c21aa2-4876-46f7-b29c-a257975dee64 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:27.887571 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c21aa2-4876-46f7-b29c-a257975dee64 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:27.887841 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c21aa2-4876-46f7-b29c-a257975dee64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:27.915573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-96c21aa2-4876-46f7-b29c-a257975dee64 None None] MFA Rules not processed for user `b158289884b944119ab956aa7e31ed1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:27.916776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-96c21aa2-4876-46f7-b29c-a257975dee64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:27.989496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1290/2575] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:27 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:28.002341 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847d91f-36a5-4ff6-a418-574d163a15bb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:28.004433 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8847d91f-36a5-4ff6-a418-574d163a15bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:28.047832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:28.068955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847d91f-36a5-4ff6-a418-574d163a15bb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:28.075629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a48fb-613b-494d-a7a3-be31c6fc2aa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:28.076459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a48fb-613b-494d-a7a3-be31c6fc2aa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:28.076827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a48fb-613b-494d-a7a3-be31c6fc2aa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:28.077204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a48fb-613b-494d-a7a3-be31c6fc2aa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:28.088144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a48fb-613b-494d-a7a3-be31c6fc2aa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:28.089594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a48fb-613b-494d-a7a3-be31c6fc2aa7 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:28.099712 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1286/2576] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:28 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:28.146615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c21aa2-4876-46f7-b29c-a257975dee64 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:28.148251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-96c21aa2-4876-46f7-b29c-a257975dee64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:28.161417 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c21aa2-4876-46f7-b29c-a257975dee64 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:28.165331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f41d2-f19e-46d6-8d7a-7cdc8eb66efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:28.166056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f41d2-f19e-46d6-8d7a-7cdc8eb66efe service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:28.166361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f41d2-f19e-46d6-8d7a-7cdc8eb66efe service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:28.166634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f41d2-f19e-46d6-8d7a-7cdc8eb66efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:28.226252 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f41d2-f19e-46d6-8d7a-7cdc8eb66efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:28.227709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f41d2-f19e-46d6-8d7a-7cdc8eb66efe service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:28.236835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1291/2577] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:30.436027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f1843-8f33-41cc-b11f-5be9b0114161 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:30.437078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f1843-8f33-41cc-b11f-5be9b0114161 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:30.437078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f1843-8f33-41cc-b11f-5be9b0114161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:30.463334 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-161f1843-8f33-41cc-b11f-5be9b0114161 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:30.464537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-161f1843-8f33-41cc-b11f-5be9b0114161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:30.525264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1287/2578] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:21:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:30.538092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f41d2-f19e-46d6-8d7a-7cdc8eb66efe service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:30.539575 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d75f41d2-f19e-46d6-8d7a-7cdc8eb66efe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:30.569757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:30.618031 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f41d2-f19e-46d6-8d7a-7cdc8eb66efe service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:30.623241 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55caebe1-fc23-429e-8493-8e1974b22ad2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:30.623910 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caebe1-fc23-429e-8493-8e1974b22ad2 service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:30.624193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caebe1-fc23-429e-8493-8e1974b22ad2 service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:30.624471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caebe1-fc23-429e-8493-8e1974b22ad2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:30.632936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55caebe1-fc23-429e-8493-8e1974b22ad2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:30.634577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55caebe1-fc23-429e-8493-8e1974b22ad2 service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:30.642969 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1292/2579] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:21:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:21:31.704231 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f1843-8f33-41cc-b11f-5be9b0114161 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:31.802491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f1843-8f33-41cc-b11f-5be9b0114161 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:31.807416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:31.808816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:31.809118 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:31.809526 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] PATH_INFO: `/v3/users/e661c3e8dd154aecb0dfc6e46c60ab75` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:31.810758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e661c3e8dd154aecb0dfc6e46c60ab75)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:31.813479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:31.878794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e661c3e8dd154aecb0dfc6e46c60ab75'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:31.879491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e661c3e8dd154aecb0dfc6e46c60ab75'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:31.880108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e661c3e8dd154aecb0dfc6e46c60ab75'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:31.891367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e661c3e8dd154aecb0dfc6e46c60ab75'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:31.901268 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e661c3e8dd154aecb0dfc6e46c60ab75'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:31.921629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1288/2580] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:31 2022] DELETE /identity/v3/users/e661c3e8dd154aecb0dfc6e46c60ab75 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:31.926783 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec60dfc-85cd-4167-aaf1-71343558a8e5 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:31.927791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec60dfc-85cd-4167-aaf1-71343558a8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:31.928445 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec60dfc-85cd-4167-aaf1-71343558a8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:31.949757 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0ec60dfc-85cd-4167-aaf1-71343558a8e5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:31.951156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0ec60dfc-85cd-4167-aaf1-71343558a8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:32.025662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1293/2581] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:32.035538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:32.038684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:32.046423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:32.108747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ed3ee-112e-4f75-830c-9a495c199cdb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:32.113137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253797d4-c9e4-4eb0-9b37-f625fca8d9c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:32.113917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253797d4-c9e4-4eb0-9b37-f625fca8d9c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:32.114315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253797d4-c9e4-4eb0-9b37-f625fca8d9c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:32.114663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253797d4-c9e4-4eb0-9b37-f625fca8d9c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:32.126528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253797d4-c9e4-4eb0-9b37-f625fca8d9c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:32.128165 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253797d4-c9e4-4eb0-9b37-f625fca8d9c1 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:32.140196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1289/2582] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:21:32.448225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec60dfc-85cd-4167-aaf1-71343558a8e5 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:32.449913 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0ec60dfc-85cd-4167-aaf1-71343558a8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:32.470454 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec60dfc-85cd-4167-aaf1-71343558a8e5 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:32.479868 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:32.480799 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:32.481054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:32.481339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] PATH_INFO: `/v3/projects/0dc41cc947d3428ab91cad237fe4982e` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:32.483436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc41cc947d3428ab91cad237fe4982e)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:32.486077 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:32.548005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc41cc947d3428ab91cad237fe4982e', 'request_id': 'req-03bedaac-d10c-4aae-8ab8-e9873f31d275'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:32.559898 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc41cc947d3428ab91cad237fe4982e', 'request_id': 'req-03bedaac-d10c-4aae-8ab8-e9873f31d275'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:32.568558 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc41cc947d3428ab91cad237fe4982e', 'request_id': 'req-03bedaac-d10c-4aae-8ab8-e9873f31d275'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:32.582213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] The token cache is being invalidate because project 0dc41cc947d3428ab91cad237fe4982e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:32.582879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:32.583808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:32.588775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1294/2583] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:32 2022] DELETE /identity/v3/projects/0dc41cc947d3428ab91cad237fe4982e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:32.589916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253797d4-c9e4-4eb0-9b37-f625fca8d9c1 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:32.602476 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253797d4-c9e4-4eb0-9b37-f625fca8d9c1 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:32.669955 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a8a30-cfbb-44e1-9dd9-550c0b9e15df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:32.671227 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a8a30-cfbb-44e1-9dd9-550c0b9e15df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:32.671485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a8a30-cfbb-44e1-9dd9-550c0b9e15df admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:32.671769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a8a30-cfbb-44e1-9dd9-550c0b9e15df admin admin] PATH_INFO: `/v3/users/b158289884b944119ab956aa7e31ed1f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:32.673372 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a8a30-cfbb-44e1-9dd9-550c0b9e15df admin admin] RBAC: Authorizing `identity:delete_user(user_id=b158289884b944119ab956aa7e31ed1f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:32.675472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a8a30-cfbb-44e1-9dd9-550c0b9e15df admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:32.733156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a5a8a30-cfbb-44e1-9dd9-550c0b9e15df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b158289884b944119ab956aa7e31ed1f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:32.734339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a5a8a30-cfbb-44e1-9dd9-550c0b9e15df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b158289884b944119ab956aa7e31ed1f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:32.734950 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a5a8a30-cfbb-44e1-9dd9-550c0b9e15df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b158289884b944119ab956aa7e31ed1f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:32.743796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a5a8a30-cfbb-44e1-9dd9-550c0b9e15df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b158289884b944119ab956aa7e31ed1f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:32.753331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a5a8a30-cfbb-44e1-9dd9-550c0b9e15df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b158289884b944119ab956aa7e31ed1f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:32.776589 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1290/2584] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:32 2022] DELETE /identity/v3/users/b158289884b944119ab956aa7e31ed1f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:33.197327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:33.199814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:33.215771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bedaac-d10c-4aae-8ab8-e9873f31d275 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:33.273193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:33.310233 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:33.311194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.311634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.311927 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] PATH_INFO: `/v3/projects/033a85a781b74c4089c90a60504fdf0c` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.313364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=033a85a781b74c4089c90a60504fdf0c)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:33.316355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:33.426118 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033a85a781b74c4089c90a60504fdf0c', 'request_id': 'req-cb1fa027-f379-4c62-bec7-dbfefcd34c87'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:33.435508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033a85a781b74c4089c90a60504fdf0c', 'request_id': 'req-cb1fa027-f379-4c62-bec7-dbfefcd34c87'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:33.445062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033a85a781b74c4089c90a60504fdf0c', 'request_id': 'req-cb1fa027-f379-4c62-bec7-dbfefcd34c87'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:33.467733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] The token cache is being invalidate because project 033a85a781b74c4089c90a60504fdf0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:33.468173 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:33.469013 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-cb1fa027-f379-4c62-bec7-dbfefcd34c87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:33.471565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1295/2585] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:33 2022] DELETE /identity/v3/projects/033a85a781b74c4089c90a60504fdf0c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:33.476413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38601fbc-f1f8-4381-9c67-3bb888a8a235 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.477061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38601fbc-f1f8-4381-9c67-3bb888a8a235 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.477515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38601fbc-f1f8-4381-9c67-3bb888a8a235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.499899 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-38601fbc-f1f8-4381-9c67-3bb888a8a235 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:33.501101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-38601fbc-f1f8-4381-9c67-3bb888a8a235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:33.525220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:33.576680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1291/2586] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:21:33.579092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e42ea0-5fd9-4d7b-8f37-d9b3dd7e9091 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.579383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e42ea0-5fd9-4d7b-8f37-d9b3dd7e9091 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.579665 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e42ea0-5fd9-4d7b-8f37-d9b3dd7e9091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.603063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-36e42ea0-5fd9-4d7b-8f37-d9b3dd7e9091 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:33.604310 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-36e42ea0-5fd9-4d7b-8f37-d9b3dd7e9091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:33.612987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1296/2587] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:33.617397 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38601fbc-f1f8-4381-9c67-3bb888a8a235 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:33.628109 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38601fbc-f1f8-4381-9c67-3bb888a8a235 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:33.634414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ac826-974b-4be5-a579-30110e1d5792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:33.635109 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac826-974b-4be5-a579-30110e1d5792 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.635393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac826-974b-4be5-a579-30110e1d5792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.635672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac826-974b-4be5-a579-30110e1d5792 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.639597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ac826-974b-4be5-a579-30110e1d5792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:21:33.639597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ac826-974b-4be5-a579-30110e1d5792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:33.639597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ac826-974b-4be5-a579-30110e1d5792 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:33.663213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1292/2588] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:21:33.666095 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e42ea0-5fd9-4d7b-8f37-d9b3dd7e9091 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:33.683028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e42ea0-5fd9-4d7b-8f37-d9b3dd7e9091 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:33.691717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1097fcf-9770-407b-913b-c955e1f07b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:33.692319 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097fcf-9770-407b-913b-c955e1f07b0a admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.692596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097fcf-9770-407b-913b-c955e1f07b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.692869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097fcf-9770-407b-913b-c955e1f07b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.693580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1097fcf-9770-407b-913b-c955e1f07b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:33.695561 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1097fcf-9770-407b-913b-c955e1f07b0a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:33.733377 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1297/2589] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:33 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:33.737783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ac826-974b-4be5-a579-30110e1d5792 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:33.757138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ac826-974b-4be5-a579-30110e1d5792 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:33.847628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b3636-d724-4dab-8ba5-b24234f04e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:33.848440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b3636-d724-4dab-8ba5-b24234f04e36 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.848762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b3636-d724-4dab-8ba5-b24234f04e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.849043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b3636-d724-4dab-8ba5-b24234f04e36 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.849758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b3636-d724-4dab-8ba5-b24234f04e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:33.851832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b3636-d724-4dab-8ba5-b24234f04e36 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:33.876655 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1293/2590] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:33 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:33.881260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1097fcf-9770-407b-913b-c955e1f07b0a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:33.893673 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1097fcf-9770-407b-913b-c955e1f07b0a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:33.898972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681e636-3c75-4ad3-a5f5-6fd125b5e014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:33.899655 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681e636-3c75-4ad3-a5f5-6fd125b5e014 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.899935 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681e636-3c75-4ad3-a5f5-6fd125b5e014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.900245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681e636-3c75-4ad3-a5f5-6fd125b5e014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.900999 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681e636-3c75-4ad3-a5f5-6fd125b5e014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:33.902668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681e636-3c75-4ad3-a5f5-6fd125b5e014 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:33.914986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1298/2591] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:33.918851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b3636-d724-4dab-8ba5-b24234f04e36 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:33.931181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b3636-d724-4dab-8ba5-b24234f04e36 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:33.937742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1dfac-0ab7-42ec-967c-3273dbd8fffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:33.938465 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1dfac-0ab7-42ec-967c-3273dbd8fffd admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.938746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1dfac-0ab7-42ec-967c-3273dbd8fffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.939021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1dfac-0ab7-42ec-967c-3273dbd8fffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:33.939771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1dfac-0ab7-42ec-967c-3273dbd8fffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:33.941678 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1dfac-0ab7-42ec-967c-3273dbd8fffd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:33.954278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1294/2592] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:33.957857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681e636-3c75-4ad3-a5f5-6fd125b5e014 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:33.973343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681e636-3c75-4ad3-a5f5-6fd125b5e014 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:33.983806 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2127b-8fd9-4536-97ff-cbb2752050b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:33.984887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2127b-8fd9-4536-97ff-cbb2752050b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:33.985244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2127b-8fd9-4536-97ff-cbb2752050b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:33.985630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2127b-8fd9-4536-97ff-cbb2752050b9 admin admin] PATH_INFO: `/v3/projects/9d74a6e639e24dbdb7f4acb340bede75/users/38101309694c414cbe358efee2e4414d/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:34.014498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2127b-8fd9-4536-97ff-cbb2752050b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d74a6e639e24dbdb7f4acb340bede75, user_id=38101309694c414cbe358efee2e4414d, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:34.017650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2127b-8fd9-4536-97ff-cbb2752050b9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:34.061411 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1299/2593] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:33 2022] PUT /identity/v3/projects/9d74a6e639e24dbdb7f4acb340bede75/users/38101309694c414cbe358efee2e4414d/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:34.061766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be7624-f9a7-4606-80b1-d965cd5967a8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:34.061766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be7624-f9a7-4606-80b1-d965cd5967a8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:34.061766 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be7624-f9a7-4606-80b1-d965cd5967a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:34.097470 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b0be7624-f9a7-4606-80b1-d965cd5967a8 None None] MFA Rules not processed for user `38101309694c414cbe358efee2e4414d`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:34.098912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b0be7624-f9a7-4606-80b1-d965cd5967a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:34.194601 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1295/2594] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:34.199130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2127b-8fd9-4536-97ff-cbb2752050b9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:34.284068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2127b-8fd9-4536-97ff-cbb2752050b9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:34.292944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6d479-ca49-4bb7-9daa-06741346e4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:34.294101 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6d479-ca49-4bb7-9daa-06741346e4fc admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:34.294651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6d479-ca49-4bb7-9daa-06741346e4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:34.295113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6d479-ca49-4bb7-9daa-06741346e4fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:34.296327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6d479-ca49-4bb7-9daa-06741346e4fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:34.301183 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6d479-ca49-4bb7-9daa-06741346e4fc admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:34.331177 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:34.363471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1300/2595] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:34 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:34.367886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be7624-f9a7-4606-80b1-d965cd5967a8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:34.379565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be7624-f9a7-4606-80b1-d965cd5967a8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:34.404500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:34.457160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751e4ad-90db-4f5f-bbca-b85f903a8a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:34.457968 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751e4ad-90db-4f5f-bbca-b85f903a8a96 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:34.458257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751e4ad-90db-4f5f-bbca-b85f903a8a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:34.458538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751e4ad-90db-4f5f-bbca-b85f903a8a96 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:34.459270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751e4ad-90db-4f5f-bbca-b85f903a8a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:34.461242 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751e4ad-90db-4f5f-bbca-b85f903a8a96 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:34.496000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1296/2596] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:34 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:34.497346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6d479-ca49-4bb7-9daa-06741346e4fc admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:34.512014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6d479-ca49-4bb7-9daa-06741346e4fc admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:34.520322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466984b-77de-4e84-bb14-9ee3c0f88f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:34.521252 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466984b-77de-4e84-bb14-9ee3c0f88f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:34.521634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466984b-77de-4e84-bb14-9ee3c0f88f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:34.522023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466984b-77de-4e84-bb14-9ee3c0f88f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:34.522837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466984b-77de-4e84-bb14-9ee3c0f88f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:34.524685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466984b-77de-4e84-bb14-9ee3c0f88f1a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:34.543489 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1301/2597] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:34.545335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751e4ad-90db-4f5f-bbca-b85f903a8a96 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:34.557622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751e4ad-90db-4f5f-bbca-b85f903a8a96 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:34.563612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2ffc1-7ba5-49c8-a953-320d620e3b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:34.564635 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2ffc1-7ba5-49c8-a953-320d620e3b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:34.564914 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2ffc1-7ba5-49c8-a953-320d620e3b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:34.565278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2ffc1-7ba5-49c8-a953-320d620e3b71 admin admin] PATH_INFO: `/v3/projects/bc091311f9aa409e87f6ae3fb00acd12/users/2f4d150b856d4caba7bb3789f95d4666/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:34.584873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2ffc1-7ba5-49c8-a953-320d620e3b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc091311f9aa409e87f6ae3fb00acd12, user_id=2f4d150b856d4caba7bb3789f95d4666, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:34.588359 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2ffc1-7ba5-49c8-a953-320d620e3b71 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:34.610306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1297/2598] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:34 2022] PUT /identity/v3/projects/bc091311f9aa409e87f6ae3fb00acd12/users/2f4d150b856d4caba7bb3789f95d4666/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:34.610844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466984b-77de-4e84-bb14-9ee3c0f88f1a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:34.622382 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466984b-77de-4e84-bb14-9ee3c0f88f1a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:34.703093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af57e1-b2ad-42fa-afdd-820cb6706ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:34.704060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af57e1-b2ad-42fa-afdd-820cb6706ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:34.704362 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af57e1-b2ad-42fa-afdd-820cb6706ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:34.704642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af57e1-b2ad-42fa-afdd-820cb6706ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:34.705379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af57e1-b2ad-42fa-afdd-820cb6706ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:34.707109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af57e1-b2ad-42fa-afdd-820cb6706ad5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:34.720690 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1302/2599] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:34.725074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2ffc1-7ba5-49c8-a953-320d620e3b71 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:34.736128 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2ffc1-7ba5-49c8-a953-320d620e3b71 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:34.743376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb0232-a850-4dad-a948-10f6a2884523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:34.744301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0232-a850-4dad-a948-10f6a2884523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:34.744584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0232-a850-4dad-a948-10f6a2884523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:34.744865 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0232-a850-4dad-a948-10f6a2884523 admin admin] PATH_INFO: `/v3/projects/bc091311f9aa409e87f6ae3fb00acd12/users/2f4d150b856d4caba7bb3789f95d4666/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:34.747028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb0232-a850-4dad-a948-10f6a2884523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc091311f9aa409e87f6ae3fb00acd12, user_id=2f4d150b856d4caba7bb3789f95d4666, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:34.749649 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb0232-a850-4dad-a948-10f6a2884523 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:34.765940 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1298/2600] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:34 2022] PUT /identity/v3/projects/bc091311f9aa409e87f6ae3fb00acd12/users/2f4d150b856d4caba7bb3789f95d4666/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:34.773409 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7569d-92c8-411d-afc2-db83d1442f02 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:34.773705 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7569d-92c8-411d-afc2-db83d1442f02 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:34.774032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7569d-92c8-411d-afc2-db83d1442f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:34.804428 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-00d7569d-92c8-411d-afc2-db83d1442f02 None None] MFA Rules not processed for user `2f4d150b856d4caba7bb3789f95d4666`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:34.807512 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-00d7569d-92c8-411d-afc2-db83d1442f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:34.898311 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1303/2601] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:34.909578 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb0232-a850-4dad-a948-10f6a2884523 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:34.911642 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-97cb0232-a850-4dad-a948-10f6a2884523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:34.979046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb0232-a850-4dad-a948-10f6a2884523 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:34.983100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae69120-e720-4ccf-a4ad-0f8f8655b153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:34.983825 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae69120-e720-4ccf-a4ad-0f8f8655b153 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:34.984180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae69120-e720-4ccf-a4ad-0f8f8655b153 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:34.984463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae69120-e720-4ccf-a4ad-0f8f8655b153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:34.996207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae69120-e720-4ccf-a4ad-0f8f8655b153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:34.997714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae69120-e720-4ccf-a4ad-0f8f8655b153 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:35.010074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1299/2602] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:35.037277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7569d-92c8-411d-afc2-db83d1442f02 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:35.104242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7569d-92c8-411d-afc2-db83d1442f02 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:35.109192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:35.110836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:35.110836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:35.110836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] PATH_INFO: `/v3/users/38101309694c414cbe358efee2e4414d` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:35.112063 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38101309694c414cbe358efee2e4414d)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:35.114526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:35.116641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:35.182128 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38101309694c414cbe358efee2e4414d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:35.183527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38101309694c414cbe358efee2e4414d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:35.195837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38101309694c414cbe358efee2e4414d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:35.204840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38101309694c414cbe358efee2e4414d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:35.205565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38101309694c414cbe358efee2e4414d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:35.227820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1304/2603] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:35 2022] DELETE /identity/v3/users/38101309694c414cbe358efee2e4414d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:35.232625 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0244dd-76f6-420d-a6a6-46b43368128f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:35.232920 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0244dd-76f6-420d-a6a6-46b43368128f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:35.233225 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0244dd-76f6-420d-a6a6-46b43368128f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:35.259129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5d0244dd-76f6-420d-a6a6-46b43368128f None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:35.260746 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5d0244dd-76f6-420d-a6a6-46b43368128f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:35.341594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1300/2604] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:35.352071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:35.356638 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:35.469377 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfd6d3-a2d3-4734-8039-a0b5872378b1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:35.474586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e1523-306f-4a87-9cd3-1b22111d4fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:35.475358 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e1523-306f-4a87-9cd3-1b22111d4fec service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:35.475744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e1523-306f-4a87-9cd3-1b22111d4fec service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:35.476345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e1523-306f-4a87-9cd3-1b22111d4fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:35.488851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e1523-306f-4a87-9cd3-1b22111d4fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:35.490653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e1523-306f-4a87-9cd3-1b22111d4fec service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:35.510858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1305/2605] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:35.850645 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0244dd-76f6-420d-a6a6-46b43368128f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:35.852648 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5d0244dd-76f6-420d-a6a6-46b43368128f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:35.874346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0244dd-76f6-420d-a6a6-46b43368128f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:35.882208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:35.883180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:35.883526 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:35.883924 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] PATH_INFO: `/v3/projects/9d74a6e639e24dbdb7f4acb340bede75` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:35.885421 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d74a6e639e24dbdb7f4acb340bede75)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:35.887449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:35.912173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:35.965775 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d74a6e639e24dbdb7f4acb340bede75', 'request_id': 'req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:35.978179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d74a6e639e24dbdb7f4acb340bede75', 'request_id': 'req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:35.985775 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d74a6e639e24dbdb7f4acb340bede75', 'request_id': 'req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:35.999375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] The token cache is being invalidate because project 9d74a6e639e24dbdb7f4acb340bede75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:35.999785 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.000360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.005805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1301/2606] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:35 2022] DELETE /identity/v3/projects/9d74a6e639e24dbdb7f4acb340bede75 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:36.006865 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e1523-306f-4a87-9cd3-1b22111d4fec service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:36.017267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e1523-306f-4a87-9cd3-1b22111d4fec service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:36.090023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb40cc-5ae4-4f6f-98a3-965887becd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:36.090943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb40cc-5ae4-4f6f-98a3-965887becd11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:36.091227 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb40cc-5ae4-4f6f-98a3-965887becd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:36.091502 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb40cc-5ae4-4f6f-98a3-965887becd11 admin admin] PATH_INFO: `/v3/users/2f4d150b856d4caba7bb3789f95d4666` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:36.092818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb40cc-5ae4-4f6f-98a3-965887becd11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f4d150b856d4caba7bb3789f95d4666)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:36.094921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb40cc-5ae4-4f6f-98a3-965887becd11 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:36.133075 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:36.169828 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4dbb40cc-5ae4-4f6f-98a3-965887becd11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f4d150b856d4caba7bb3789f95d4666'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.170562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4dbb40cc-5ae4-4f6f-98a3-965887becd11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f4d150b856d4caba7bb3789f95d4666'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.184145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4dbb40cc-5ae4-4f6f-98a3-965887becd11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f4d150b856d4caba7bb3789f95d4666'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.194449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4dbb40cc-5ae4-4f6f-98a3-965887becd11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f4d150b856d4caba7bb3789f95d4666'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.195213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-4dbb40cc-5ae4-4f6f-98a3-965887becd11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f4d150b856d4caba7bb3789f95d4666'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.215351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1306/2607] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:36 2022] DELETE /identity/v3/users/2f4d150b856d4caba7bb3789f95d4666 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:36.561500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:36.563026 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:36.587009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77076d8-ecd4-4c36-b8b9-ab2be1ae0a4c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:36.694502 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:36.695445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:36.695730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:36.696001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] PATH_INFO: `/v3/projects/bc091311f9aa409e87f6ae3fb00acd12` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:36.697851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc091311f9aa409e87f6ae3fb00acd12)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:36.699362 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:36.771033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc091311f9aa409e87f6ae3fb00acd12', 'request_id': 'req-08985092-4af6-4667-a4d6-2216f1ce8dea'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.800192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc091311f9aa409e87f6ae3fb00acd12', 'request_id': 'req-08985092-4af6-4667-a4d6-2216f1ce8dea'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.814443 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc091311f9aa409e87f6ae3fb00acd12', 'request_id': 'req-08985092-4af6-4667-a4d6-2216f1ce8dea'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.829506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] The token cache is being invalidate because project bc091311f9aa409e87f6ae3fb00acd12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:36.829506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.830695 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08985092-4af6-4667-a4d6-2216f1ce8dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:36.832975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1302/2608] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:36 2022] DELETE /identity/v3/projects/bc091311f9aa409e87f6ae3fb00acd12 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:36.838696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496c624-168f-412a-9ff2-706342053903 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:36.839152 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496c624-168f-412a-9ff2-706342053903 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:36.839469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496c624-168f-412a-9ff2-706342053903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:36.860415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7496c624-168f-412a-9ff2-706342053903 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:36.861790 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7496c624-168f-412a-9ff2-706342053903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:36.933691 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1307/2609] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:36.942724 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f127cd8-7ad0-4243-8058-c49753ed367e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:36.943322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f127cd8-7ad0-4243-8058-c49753ed367e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:36.943322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f127cd8-7ad0-4243-8058-c49753ed367e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:36.964346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9f127cd8-7ad0-4243-8058-c49753ed367e None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:36.965519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9f127cd8-7ad0-4243-8058-c49753ed367e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:36.978150 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1303/2610] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:36.982740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496c624-168f-412a-9ff2-706342053903 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:36.993174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496c624-168f-412a-9ff2-706342053903 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:36.999470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f46be3-318e-410c-ac51-704ed0ab2a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:37.000156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46be3-318e-410c-ac51-704ed0ab2a82 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:37.000440 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46be3-318e-410c-ac51-704ed0ab2a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:37.000731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f46be3-318e-410c-ac51-704ed0ab2a82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:37.001468 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f46be3-318e-410c-ac51-704ed0ab2a82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:21:37.001777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f46be3-318e-410c-ac51-704ed0ab2a82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:37.003659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f46be3-318e-410c-ac51-704ed0ab2a82 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:37.023553 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1308/2611] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:21:37.024529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f127cd8-7ad0-4243-8058-c49753ed367e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:37.034980 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f127cd8-7ad0-4243-8058-c49753ed367e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:37.042357 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107300e-30e2-4c32-9580-51356658121e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:37.043309 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107300e-30e2-4c32-9580-51356658121e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:37.043838 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107300e-30e2-4c32-9580-51356658121e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:37.044117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107300e-30e2-4c32-9580-51356658121e admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:37.045079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107300e-30e2-4c32-9580-51356658121e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:37.047107 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107300e-30e2-4c32-9580-51356658121e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:37.082648 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f46be3-318e-410c-ac51-704ed0ab2a82 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:37.083824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1304/2612] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:37 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:21:37.095323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f46be3-318e-410c-ac51-704ed0ab2a82 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:37.175791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02d325-ca39-441f-9ee8-1211cdc00c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:37.176616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d325-ca39-441f-9ee8-1211cdc00c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:37.176979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d325-ca39-441f-9ee8-1211cdc00c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:37.177327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d325-ca39-441f-9ee8-1211cdc00c1f admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:37.178343 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02d325-ca39-441f-9ee8-1211cdc00c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:37.180789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02d325-ca39-441f-9ee8-1211cdc00c1f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:37.206355 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1309/2613] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:37 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:37.212290 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107300e-30e2-4c32-9580-51356658121e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:37.219363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:37.226664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107300e-30e2-4c32-9580-51356658121e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:37.233608 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582ba4b-3406-491f-9d4a-6ad96c99de2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:37.234739 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582ba4b-3406-491f-9d4a-6ad96c99de2c admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:37.235116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582ba4b-3406-491f-9d4a-6ad96c99de2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:37.235517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582ba4b-3406-491f-9d4a-6ad96c99de2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:37.236374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582ba4b-3406-491f-9d4a-6ad96c99de2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:37.238780 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582ba4b-3406-491f-9d4a-6ad96c99de2c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:37.253848 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1305/2614] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:37.258231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02d325-ca39-441f-9ee8-1211cdc00c1f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:37.270454 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02d325-ca39-441f-9ee8-1211cdc00c1f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:37.275247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de2cff-e795-4489-82fa-ad9ab17fa4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:37.276057 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de2cff-e795-4489-82fa-ad9ab17fa4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:37.276372 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de2cff-e795-4489-82fa-ad9ab17fa4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:37.276688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de2cff-e795-4489-82fa-ad9ab17fa4f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:37.277532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de2cff-e795-4489-82fa-ad9ab17fa4f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:37.279303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de2cff-e795-4489-82fa-ad9ab17fa4f0 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:37.291845 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1310/2615] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:37.295989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582ba4b-3406-491f-9d4a-6ad96c99de2c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:37.308674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582ba4b-3406-491f-9d4a-6ad96c99de2c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:37.313747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa0879-9f85-41f7-89b5-40ead5acb3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:37.314751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa0879-9f85-41f7-89b5-40ead5acb3cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:37.315021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa0879-9f85-41f7-89b5-40ead5acb3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:37.315779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa0879-9f85-41f7-89b5-40ead5acb3cf admin admin] PATH_INFO: `/v3/projects/8f39c4053c8c4a799f3e11748519a401/users/466871dbd2064f9abf5388cd2124849f/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:37.335833 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa0879-9f85-41f7-89b5-40ead5acb3cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f39c4053c8c4a799f3e11748519a401, user_id=466871dbd2064f9abf5388cd2124849f, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:37.338874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa0879-9f85-41f7-89b5-40ead5acb3cf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:37.359864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1306/2616] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:37 2022] PUT /identity/v3/projects/8f39c4053c8c4a799f3e11748519a401/users/466871dbd2064f9abf5388cd2124849f/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:37.374764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be36138-a95c-4706-a340-c81b10d90ce8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:37.386591 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be36138-a95c-4706-a340-c81b10d90ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:37.386993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be36138-a95c-4706-a340-c81b10d90ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:37.407906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:37.426077 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-7be36138-a95c-4706-a340-c81b10d90ce8 None None] MFA Rules not processed for user `466871dbd2064f9abf5388cd2124849f`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:37.427659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7be36138-a95c-4706-a340-c81b10d90ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:37.504165 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1311/2617] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:37.508897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa0879-9f85-41f7-89b5-40ead5acb3cf admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:37.592553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa0879-9f85-41f7-89b5-40ead5acb3cf admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:37.605827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b328f-2353-4054-abc4-a6dbc9f9aa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:37.606692 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b328f-2353-4054-abc4-a6dbc9f9aa5c admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:37.607039 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b328f-2353-4054-abc4-a6dbc9f9aa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:37.607383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b328f-2353-4054-abc4-a6dbc9f9aa5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:37.608313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b328f-2353-4054-abc4-a6dbc9f9aa5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:37.612033 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b328f-2353-4054-abc4-a6dbc9f9aa5c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:37.645332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1307/2618] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:37 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:21:37.646743 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be36138-a95c-4706-a340-c81b10d90ce8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:37.660168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be36138-a95c-4706-a340-c81b10d90ce8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:37.733573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd05d52-c431-4fa4-a862-e329aad6b65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:37.734449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd05d52-c431-4fa4-a862-e329aad6b65c admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:37.734782 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd05d52-c431-4fa4-a862-e329aad6b65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:37.735143 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd05d52-c431-4fa4-a862-e329aad6b65c admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:37.735968 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd05d52-c431-4fa4-a862-e329aad6b65c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:37.738121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd05d52-c431-4fa4-a862-e329aad6b65c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:37.763993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1312/2619] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:37 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:37.768706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b328f-2353-4054-abc4-a6dbc9f9aa5c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:37.781442 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b328f-2353-4054-abc4-a6dbc9f9aa5c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:37.795909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68af80d-dac4-4c07-ab99-1f49ad6ed4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:37.796396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68af80d-dac4-4c07-ab99-1f49ad6ed4df admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:37.797875 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68af80d-dac4-4c07-ab99-1f49ad6ed4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:37.798169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68af80d-dac4-4c07-ab99-1f49ad6ed4df admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:37.798940 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68af80d-dac4-4c07-ab99-1f49ad6ed4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:37.800765 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68af80d-dac4-4c07-ab99-1f49ad6ed4df admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:37.816324 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1308/2620] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:37.820465 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd05d52-c431-4fa4-a862-e329aad6b65c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:37.834102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd05d52-c431-4fa4-a862-e329aad6b65c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:37.841670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40cc07-eeed-4dba-b9d9-3228e5190f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:37.842686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40cc07-eeed-4dba-b9d9-3228e5190f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:37.843023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40cc07-eeed-4dba-b9d9-3228e5190f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:37.843439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40cc07-eeed-4dba-b9d9-3228e5190f58 admin admin] PATH_INFO: `/v3/projects/b5d20cfc19424de5bdcf9bfb851e5042/users/9dbaaf4f4dde4ed4affbfdc915436e9b/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:37.865487 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40cc07-eeed-4dba-b9d9-3228e5190f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d20cfc19424de5bdcf9bfb851e5042, user_id=9dbaaf4f4dde4ed4affbfdc915436e9b, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:37.868383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40cc07-eeed-4dba-b9d9-3228e5190f58 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:37.904871 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1313/2621] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:37 2022] PUT /identity/v3/projects/b5d20cfc19424de5bdcf9bfb851e5042/users/9dbaaf4f4dde4ed4affbfdc915436e9b/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:37.908812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68af80d-dac4-4c07-ab99-1f49ad6ed4df admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:37.929972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68af80d-dac4-4c07-ab99-1f49ad6ed4df admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:38.010719 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f591f3e-53cd-4612-98ec-378fab8fd7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:38.011513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f591f3e-53cd-4612-98ec-378fab8fd7da admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:38.011942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f591f3e-53cd-4612-98ec-378fab8fd7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:38.012292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f591f3e-53cd-4612-98ec-378fab8fd7da admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:38.013162 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f591f3e-53cd-4612-98ec-378fab8fd7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:38.015185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f591f3e-53cd-4612-98ec-378fab8fd7da admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:38.033611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40cc07-eeed-4dba-b9d9-3228e5190f58 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:38.043993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1309/2622] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:38.055053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40cc07-eeed-4dba-b9d9-3228e5190f58 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:38.060182 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d288ebf-34a0-42ff-a1e9-35b44428128c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:38.061114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d288ebf-34a0-42ff-a1e9-35b44428128c admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:38.061399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d288ebf-34a0-42ff-a1e9-35b44428128c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:38.061687 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d288ebf-34a0-42ff-a1e9-35b44428128c admin admin] PATH_INFO: `/v3/projects/b5d20cfc19424de5bdcf9bfb851e5042/users/9dbaaf4f4dde4ed4affbfdc915436e9b/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:38.063725 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d288ebf-34a0-42ff-a1e9-35b44428128c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d20cfc19424de5bdcf9bfb851e5042, user_id=9dbaaf4f4dde4ed4affbfdc915436e9b, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:38.067989 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d288ebf-34a0-42ff-a1e9-35b44428128c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:38.099081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef856a8-edd6-4e34-abeb-5282ce4b34aa None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:38.099390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef856a8-edd6-4e34-abeb-5282ce4b34aa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:38.099672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef856a8-edd6-4e34-abeb-5282ce4b34aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:38.101625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1314/2623] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:38 2022] PUT /identity/v3/projects/b5d20cfc19424de5bdcf9bfb851e5042/users/9dbaaf4f4dde4ed4affbfdc915436e9b/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:21:38.135663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4ef856a8-edd6-4e34-abeb-5282ce4b34aa None None] MFA Rules not processed for user `9dbaaf4f4dde4ed4affbfdc915436e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:38.136935 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4ef856a8-edd6-4e34-abeb-5282ce4b34aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:38.178819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:38.240894 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1310/2624] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:38.255860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d288ebf-34a0-42ff-a1e9-35b44428128c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:38.257600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7d288ebf-34a0-42ff-a1e9-35b44428128c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:38.341082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:38.345842 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d288ebf-34a0-42ff-a1e9-35b44428128c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:38.352788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d497e1-b2d4-474b-a266-4806583c8171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:38.353611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d497e1-b2d4-474b-a266-4806583c8171 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:38.354219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d497e1-b2d4-474b-a266-4806583c8171 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:38.354717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d497e1-b2d4-474b-a266-4806583c8171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:38.365875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d497e1-b2d4-474b-a266-4806583c8171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:38.367516 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d497e1-b2d4-474b-a266-4806583c8171 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:38.378432 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1315/2625] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:38.548133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef856a8-edd6-4e34-abeb-5282ce4b34aa None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:38.634046 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef856a8-edd6-4e34-abeb-5282ce4b34aa None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:38.640823 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:38.641855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:38.642312 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:38.643143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] PATH_INFO: `/v3/users/466871dbd2064f9abf5388cd2124849f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:38.644582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=466871dbd2064f9abf5388cd2124849f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:38.646709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:38.706377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '466871dbd2064f9abf5388cd2124849f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:38.709051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '466871dbd2064f9abf5388cd2124849f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:38.711002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '466871dbd2064f9abf5388cd2124849f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:38.726404 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '466871dbd2064f9abf5388cd2124849f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:38.735738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '466871dbd2064f9abf5388cd2124849f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:38.763824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1311/2626] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:38 2022] DELETE /identity/v3/users/466871dbd2064f9abf5388cd2124849f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:38.771124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54a5a6-6b0b-43ab-8709-bc7604722775 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:38.771711 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54a5a6-6b0b-43ab-8709-bc7604722775 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:38.772145 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54a5a6-6b0b-43ab-8709-bc7604722775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:38.793527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1f54a5a6-6b0b-43ab-8709-bc7604722775 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:38.794982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1f54a5a6-6b0b-43ab-8709-bc7604722775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:38.865246 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1316/2627] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:38.875848 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:38.877864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:38.933235 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:38.951242 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82823d2e-f6de-4f30-ac00-92af07977cd1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:38.952782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442479e-b977-4e7c-8116-0b684a883366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:38.953484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442479e-b977-4e7c-8116-0b684a883366 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:38.953778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442479e-b977-4e7c-8116-0b684a883366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:38.954100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442479e-b977-4e7c-8116-0b684a883366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:38.965266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442479e-b977-4e7c-8116-0b684a883366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:38.966859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442479e-b977-4e7c-8116-0b684a883366 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:38.980413 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1312/2628] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:39.397825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54a5a6-6b0b-43ab-8709-bc7604722775 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:39.399534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1f54a5a6-6b0b-43ab-8709-bc7604722775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:39.418450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54a5a6-6b0b-43ab-8709-bc7604722775 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:39.424904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:39.426176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:39.426176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:39.427132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] PATH_INFO: `/v3/projects/8f39c4053c8c4a799f3e11748519a401` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:39.428420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f39c4053c8c4a799f3e11748519a401)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:39.431076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:39.509444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f39c4053c8c4a799f3e11748519a401', 'request_id': 'req-b38e5e71-b218-45c7-a402-676319ac3dc4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:39.522894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f39c4053c8c4a799f3e11748519a401', 'request_id': 'req-b38e5e71-b218-45c7-a402-676319ac3dc4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:39.531403 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f39c4053c8c4a799f3e11748519a401', 'request_id': 'req-b38e5e71-b218-45c7-a402-676319ac3dc4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:39.550667 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] The token cache is being invalidate because project 8f39c4053c8c4a799f3e11748519a401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:39.551047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:39.551762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:39.553587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1317/2629] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:39 2022] DELETE /identity/v3/projects/8f39c4053c8c4a799f3e11748519a401 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:39.557985 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442479e-b977-4e7c-8116-0b684a883366 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:39.568738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442479e-b977-4e7c-8116-0b684a883366 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:39.636647 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e710c-fa45-495e-87d6-b84aa3c1d942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:39.637555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e710c-fa45-495e-87d6-b84aa3c1d942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:39.637839 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e710c-fa45-495e-87d6-b84aa3c1d942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:39.638163 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e710c-fa45-495e-87d6-b84aa3c1d942 admin admin] PATH_INFO: `/v3/users/9dbaaf4f4dde4ed4affbfdc915436e9b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:39.639512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e710c-fa45-495e-87d6-b84aa3c1d942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dbaaf4f4dde4ed4affbfdc915436e9b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:39.641580 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e710c-fa45-495e-87d6-b84aa3c1d942 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:39.694701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ea1e710c-fa45-495e-87d6-b84aa3c1d942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dbaaf4f4dde4ed4affbfdc915436e9b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:39.695418 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ea1e710c-fa45-495e-87d6-b84aa3c1d942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dbaaf4f4dde4ed4affbfdc915436e9b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:39.696045 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ea1e710c-fa45-495e-87d6-b84aa3c1d942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dbaaf4f4dde4ed4affbfdc915436e9b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:39.705538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ea1e710c-fa45-495e-87d6-b84aa3c1d942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dbaaf4f4dde4ed4affbfdc915436e9b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:39.716568 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ea1e710c-fa45-495e-87d6-b84aa3c1d942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dbaaf4f4dde4ed4affbfdc915436e9b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:39.735891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1313/2630] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:39 2022] DELETE /identity/v3/users/9dbaaf4f4dde4ed4affbfdc915436e9b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:40.036376 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:40.038040 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:40.054947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e5e71-b218-45c7-a402-676319ac3dc4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:40.118511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:40.119389 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.119669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.119962 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] PATH_INFO: `/v3/projects/b5d20cfc19424de5bdcf9bfb851e5042` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.121219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5d20cfc19424de5bdcf9bfb851e5042)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:40.123180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:40.140696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:40.176972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5d20cfc19424de5bdcf9bfb851e5042', 'request_id': 'req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:40.185407 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5d20cfc19424de5bdcf9bfb851e5042', 'request_id': 'req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:40.192669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5d20cfc19424de5bdcf9bfb851e5042', 'request_id': 'req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:40.203065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] The token cache is being invalidate because project b5d20cfc19424de5bdcf9bfb851e5042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:40.203350 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:40.204016 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-47d382d0-bffa-4e89-97b6-1cf7a95b6dfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:40.205874 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1318/2631] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:40 2022] DELETE /identity/v3/projects/b5d20cfc19424de5bdcf9bfb851e5042 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:21:40.211384 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d294dd1-ffd7-4c1d-aa92-9837f25b996c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.211672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d294dd1-ffd7-4c1d-aa92-9837f25b996c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.211945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d294dd1-ffd7-4c1d-aa92-9837f25b996c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.230355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1d294dd1-ffd7-4c1d-aa92-9837f25b996c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:40.231588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d294dd1-ffd7-4c1d-aa92-9837f25b996c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:40.298281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1314/2632] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:40.306635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec676c8e-895d-460b-9533-8efaedc217f8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.306928 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec676c8e-895d-460b-9533-8efaedc217f8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.307208 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec676c8e-895d-460b-9533-8efaedc217f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.327021 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-ec676c8e-895d-460b-9533-8efaedc217f8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:40.328254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ec676c8e-895d-460b-9533-8efaedc217f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:40.336339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1319/2633] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:40.340646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d294dd1-ffd7-4c1d-aa92-9837f25b996c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:40.347488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:40.351870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d294dd1-ffd7-4c1d-aa92-9837f25b996c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:40.358101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb449c3-aa54-4ffb-9f30-9d0bb3375318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:40.358925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb449c3-aa54-4ffb-9f30-9d0bb3375318 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.359472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb449c3-aa54-4ffb-9f30-9d0bb3375318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.359942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb449c3-aa54-4ffb-9f30-9d0bb3375318 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.360843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb449c3-aa54-4ffb-9f30-9d0bb3375318 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:21:40.361403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb449c3-aa54-4ffb-9f30-9d0bb3375318 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:40.364594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb449c3-aa54-4ffb-9f30-9d0bb3375318 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:40.381092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1315/2634] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:21:40.385437 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec676c8e-895d-460b-9533-8efaedc217f8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:40.395823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec676c8e-895d-460b-9533-8efaedc217f8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:40.400537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170a7d2-009b-4077-b59f-ff6a5bebc709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:40.401201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a7d2-009b-4077-b59f-ff6a5bebc709 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.401678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a7d2-009b-4077-b59f-ff6a5bebc709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.401995 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a7d2-009b-4077-b59f-ff6a5bebc709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.402698 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170a7d2-009b-4077-b59f-ff6a5bebc709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:40.404607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170a7d2-009b-4077-b59f-ff6a5bebc709 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:40.433088 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1320/2635] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:40 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:40.437441 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb449c3-aa54-4ffb-9f30-9d0bb3375318 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:40.447839 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb449c3-aa54-4ffb-9f30-9d0bb3375318 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:40.524869 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701e5fc-3f75-45c9-8019-f677cec7ecb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:40.525847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701e5fc-3f75-45c9-8019-f677cec7ecb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.525998 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701e5fc-3f75-45c9-8019-f677cec7ecb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.526278 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701e5fc-3f75-45c9-8019-f677cec7ecb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.527040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701e5fc-3f75-45c9-8019-f677cec7ecb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:40.529169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701e5fc-3f75-45c9-8019-f677cec7ecb6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:40.553718 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1316/2636] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:40 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:40.555283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170a7d2-009b-4077-b59f-ff6a5bebc709 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:40.567173 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170a7d2-009b-4077-b59f-ff6a5bebc709 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:40.572395 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f2691-158b-4b63-a922-a75cad365cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:40.573070 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f2691-158b-4b63-a922-a75cad365cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.573351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f2691-158b-4b63-a922-a75cad365cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.573618 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f2691-158b-4b63-a922-a75cad365cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.574560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f2691-158b-4b63-a922-a75cad365cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:40.576189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f2691-158b-4b63-a922-a75cad365cf1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:40.588488 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1321/2637] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:40.592715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701e5fc-3f75-45c9-8019-f677cec7ecb6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:40.604672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701e5fc-3f75-45c9-8019-f677cec7ecb6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:40.610505 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf6f67-721f-4cf4-8ed1-9447fe071e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:40.611310 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf6f67-721f-4cf4-8ed1-9447fe071e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.611583 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf6f67-721f-4cf4-8ed1-9447fe071e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.611872 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf6f67-721f-4cf4-8ed1-9447fe071e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.612777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf6f67-721f-4cf4-8ed1-9447fe071e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:40.614338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf6f67-721f-4cf4-8ed1-9447fe071e2b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:40.629744 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1317/2638] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:40.631114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f2691-158b-4b63-a922-a75cad365cf1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:40.643968 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f2691-158b-4b63-a922-a75cad365cf1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:40.649214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9e667-ba6a-4e11-82e7-a007895dfa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:40.650251 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9e667-ba6a-4e11-82e7-a007895dfa2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.650562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9e667-ba6a-4e11-82e7-a007895dfa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.650840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9e667-ba6a-4e11-82e7-a007895dfa2d admin admin] PATH_INFO: `/v3/projects/bd56c14815644b57973337982f3973f8/users/ded5a6ec86da468e9ca7a3f29d42f99d/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.666989 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9e667-ba6a-4e11-82e7-a007895dfa2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd56c14815644b57973337982f3973f8, user_id=ded5a6ec86da468e9ca7a3f29d42f99d, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:40.669647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9e667-ba6a-4e11-82e7-a007895dfa2d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:40.685209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1322/2639] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:40 2022] PUT /identity/v3/projects/bd56c14815644b57973337982f3973f8/users/ded5a6ec86da468e9ca7a3f29d42f99d/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:40.696898 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd766476-7cc1-4b8b-ac8a-496413cb2066 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.697288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd766476-7cc1-4b8b-ac8a-496413cb2066 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.697664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd766476-7cc1-4b8b-ac8a-496413cb2066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.727591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-fd766476-7cc1-4b8b-ac8a-496413cb2066 None None] MFA Rules not processed for user `ded5a6ec86da468e9ca7a3f29d42f99d`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:40.728923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fd766476-7cc1-4b8b-ac8a-496413cb2066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:40.797674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1318/2640] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:21:40.802306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9e667-ba6a-4e11-82e7-a007895dfa2d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:40.866792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9e667-ba6a-4e11-82e7-a007895dfa2d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:40.872415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375972d-d3b8-44a2-9b68-52a2f0a452c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:40.873337 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375972d-d3b8-44a2-9b68-52a2f0a452c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.873607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375972d-d3b8-44a2-9b68-52a2f0a452c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.874035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375972d-d3b8-44a2-9b68-52a2f0a452c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.874829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375972d-d3b8-44a2-9b68-52a2f0a452c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:40.876908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375972d-d3b8-44a2-9b68-52a2f0a452c8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:40.908199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1323/2641] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:40 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:40.912257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd766476-7cc1-4b8b-ac8a-496413cb2066 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:40.923510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd766476-7cc1-4b8b-ac8a-496413cb2066 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:40.982828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9011cbc-cf5c-4992-863d-845ddca6f15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:40.983545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9011cbc-cf5c-4992-863d-845ddca6f15b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:40.983827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9011cbc-cf5c-4992-863d-845ddca6f15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:40.984068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9011cbc-cf5c-4992-863d-845ddca6f15b admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:40.984799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9011cbc-cf5c-4992-863d-845ddca6f15b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:40.986779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9011cbc-cf5c-4992-863d-845ddca6f15b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:40.993751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:41.010778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1319/2642] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:40 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:41.014712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375972d-d3b8-44a2-9b68-52a2f0a452c8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:41.026157 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375972d-d3b8-44a2-9b68-52a2f0a452c8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:41.031314 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e119bb-dd13-4cd0-ae5b-69b4210e0b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:41.032037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e119bb-dd13-4cd0-ae5b-69b4210e0b06 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:41.032316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e119bb-dd13-4cd0-ae5b-69b4210e0b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:41.032592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e119bb-dd13-4cd0-ae5b-69b4210e0b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:41.033318 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e119bb-dd13-4cd0-ae5b-69b4210e0b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:41.035068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e119bb-dd13-4cd0-ae5b-69b4210e0b06 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:41.046650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1324/2643] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:41.051305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9011cbc-cf5c-4992-863d-845ddca6f15b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:41.061770 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9011cbc-cf5c-4992-863d-845ddca6f15b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:41.067141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966db39-b91e-4e98-ba09-256e5a9036f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:41.068056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966db39-b91e-4e98-ba09-256e5a9036f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:41.068337 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966db39-b91e-4e98-ba09-256e5a9036f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:41.068613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966db39-b91e-4e98-ba09-256e5a9036f8 admin admin] PATH_INFO: `/v3/projects/d0446f8edd864a3fac0a53449539b259/users/af65aeb53d2b45028b6bcda6874ada36/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:41.083734 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966db39-b91e-4e98-ba09-256e5a9036f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0446f8edd864a3fac0a53449539b259, user_id=af65aeb53d2b45028b6bcda6874ada36, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:41.086539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966db39-b91e-4e98-ba09-256e5a9036f8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:41.101612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1320/2644] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:41 2022] PUT /identity/v3/projects/d0446f8edd864a3fac0a53449539b259/users/af65aeb53d2b45028b6bcda6874ada36/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:41.105574 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e119bb-dd13-4cd0-ae5b-69b4210e0b06 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:41.116230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e119bb-dd13-4cd0-ae5b-69b4210e0b06 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:41.133028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:41.201289 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693529d-d04c-42bb-a39e-aec075620ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:41.202006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693529d-d04c-42bb-a39e-aec075620ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:41.202507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693529d-d04c-42bb-a39e-aec075620ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:41.202794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693529d-d04c-42bb-a39e-aec075620ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:41.203535 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693529d-d04c-42bb-a39e-aec075620ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:41.205162 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693529d-d04c-42bb-a39e-aec075620ae3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:41.216516 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1325/2645] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:21:41.220895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966db39-b91e-4e98-ba09-256e5a9036f8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:41.231552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966db39-b91e-4e98-ba09-256e5a9036f8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:41.236119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279f683-575b-4075-aaf7-b690fec0ba2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:41.237093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279f683-575b-4075-aaf7-b690fec0ba2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:41.237382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279f683-575b-4075-aaf7-b690fec0ba2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:41.237658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279f683-575b-4075-aaf7-b690fec0ba2d admin admin] PATH_INFO: `/v3/projects/d0446f8edd864a3fac0a53449539b259/users/af65aeb53d2b45028b6bcda6874ada36/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:41.239930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279f683-575b-4075-aaf7-b690fec0ba2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0446f8edd864a3fac0a53449539b259, user_id=af65aeb53d2b45028b6bcda6874ada36, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:41.242623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279f683-575b-4075-aaf7-b690fec0ba2d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:41.253189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1321/2646] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:41 2022] PUT /identity/v3/projects/d0446f8edd864a3fac0a53449539b259/users/af65aeb53d2b45028b6bcda6874ada36/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:41.264220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51482f-fc7b-42a2-9a54-ec31779b91fe None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:41.264513 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51482f-fc7b-42a2-9a54-ec31779b91fe None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:41.265413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51482f-fc7b-42a2-9a54-ec31779b91fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:41.294872 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4d51482f-fc7b-42a2-9a54-ec31779b91fe None None] MFA Rules not processed for user `af65aeb53d2b45028b6bcda6874ada36`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:41.296121 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4d51482f-fc7b-42a2-9a54-ec31779b91fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:41.369964 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1326/2647] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:41.381844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279f683-575b-4075-aaf7-b690fec0ba2d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:41.383858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1279f683-575b-4075-aaf7-b690fec0ba2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:41.443667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279f683-575b-4075-aaf7-b690fec0ba2d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:41.447808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62833d7b-b09f-43f3-ac21-39c5ab368d9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:41.448452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62833d7b-b09f-43f3-ac21-39c5ab368d9c service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:41.448735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62833d7b-b09f-43f3-ac21-39c5ab368d9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:41.449014 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62833d7b-b09f-43f3-ac21-39c5ab368d9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:41.458931 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62833d7b-b09f-43f3-ac21-39c5ab368d9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:41.460400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62833d7b-b09f-43f3-ac21-39c5ab368d9c service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:41.470480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1322/2648] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:41.569634 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51482f-fc7b-42a2-9a54-ec31779b91fe None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:41.585207 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51482f-fc7b-42a2-9a54-ec31779b91fe None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:41.591775 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f2de4-38dd-4ba4-8da1-6549a33fb4fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:41.592640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f2de4-38dd-4ba4-8da1-6549a33fb4fa service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:41.592936 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f2de4-38dd-4ba4-8da1-6549a33fb4fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:41.593352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f2de4-38dd-4ba4-8da1-6549a33fb4fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:41.668636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f2de4-38dd-4ba4-8da1-6549a33fb4fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:41.670434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f2de4-38dd-4ba4-8da1-6549a33fb4fa service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:41.681700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1327/2649] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:41.957508 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62833d7b-b09f-43f3-ac21-39c5ab368d9c service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:42.026532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62833d7b-b09f-43f3-ac21-39c5ab368d9c service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:42.031510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:42.032389 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:42.032675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:42.032948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] PATH_INFO: `/v3/users/ded5a6ec86da468e9ca7a3f29d42f99d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:42.034435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded5a6ec86da468e9ca7a3f29d42f99d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:42.036358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:42.087182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded5a6ec86da468e9ca7a3f29d42f99d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.088226 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded5a6ec86da468e9ca7a3f29d42f99d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.089135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded5a6ec86da468e9ca7a3f29d42f99d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.100387 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded5a6ec86da468e9ca7a3f29d42f99d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.102574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=6,keystone:SELECT=86,keystone:DELETE=8 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:42.109237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded5a6ec86da468e9ca7a3f29d42f99d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.128670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1323/2650] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:41 2022] DELETE /identity/v3/users/ded5a6ec86da468e9ca7a3f29d42f99d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:21:42.133517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dba5f2-c252-416a-bfda-20edf4db7c3d None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:42.133806 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dba5f2-c252-416a-bfda-20edf4db7c3d None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:42.134114 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dba5f2-c252-416a-bfda-20edf4db7c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:42.152836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c4dba5f2-c252-416a-bfda-20edf4db7c3d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:42.154076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c4dba5f2-c252-416a-bfda-20edf4db7c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:42.188617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:42.218728 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1328/2651] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:21:42.228741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:42.230289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:42.289815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b2951-af6d-4249-b110-e6989c8aa419 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:42.294398 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7793f6-1762-4fbc-8ba8-35ed8d234b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:42.295044 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7793f6-1762-4fbc-8ba8-35ed8d234b29 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:42.295330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7793f6-1762-4fbc-8ba8-35ed8d234b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:42.295605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7793f6-1762-4fbc-8ba8-35ed8d234b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:42.305484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7793f6-1762-4fbc-8ba8-35ed8d234b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:42.306980 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7793f6-1762-4fbc-8ba8-35ed8d234b29 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:42.316780 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1324/2652] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:42.608153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dba5f2-c252-416a-bfda-20edf4db7c3d None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:42.609688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c4dba5f2-c252-416a-bfda-20edf4db7c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:42.624762 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dba5f2-c252-416a-bfda-20edf4db7c3d None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:42.630947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:42.631905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:42.632210 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:42.632470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] PATH_INFO: `/v3/projects/bd56c14815644b57973337982f3973f8` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:42.633712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd56c14815644b57973337982f3973f8)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:42.635761 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:42.687703 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd56c14815644b57973337982f3973f8', 'request_id': 'req-2aa67402-e485-4908-9c98-ae8aa9bf564a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.697268 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd56c14815644b57973337982f3973f8', 'request_id': 'req-2aa67402-e485-4908-9c98-ae8aa9bf564a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.703632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd56c14815644b57973337982f3973f8', 'request_id': 'req-2aa67402-e485-4908-9c98-ae8aa9bf564a'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.714273 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] The token cache is being invalidate because project bd56c14815644b57973337982f3973f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:42.714564 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.715194 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.716816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1329/2653] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:42 2022] DELETE /identity/v3/projects/bd56c14815644b57973337982f3973f8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:42.721038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7793f6-1762-4fbc-8ba8-35ed8d234b29 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:42.731204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7793f6-1762-4fbc-8ba8-35ed8d234b29 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:42.791932 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:42.792803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:42.793104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:42.793368 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] PATH_INFO: `/v3/users/af65aeb53d2b45028b6bcda6874ada36` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:42.794660 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af65aeb53d2b45028b6bcda6874ada36)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:42.796623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:42.846466 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af65aeb53d2b45028b6bcda6874ada36'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.847162 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af65aeb53d2b45028b6bcda6874ada36'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.847784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af65aeb53d2b45028b6bcda6874ada36'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.857037 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af65aeb53d2b45028b6bcda6874ada36'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.864463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af65aeb53d2b45028b6bcda6874ada36'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:42.875377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:42.883994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1325/2654] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:42 2022] DELETE /identity/v3/users/af65aeb53d2b45028b6bcda6874ada36 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:43.237540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:43.239911 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:43.253286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa67402-e485-4908-9c98-ae8aa9bf564a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:43.316154 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:43.317036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:43.317316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:43.317596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] PATH_INFO: `/v3/projects/d0446f8edd864a3fac0a53449539b259` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:43.318889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0446f8edd864a3fac0a53449539b259)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:43.320797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:43.334911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:43.336397 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:43.369940 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0446f8edd864a3fac0a53449539b259', 'request_id': 'req-549ef513-3291-440e-b8b8-62cac5876382'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:43.379857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0446f8edd864a3fac0a53449539b259', 'request_id': 'req-549ef513-3291-440e-b8b8-62cac5876382'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:43.387093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0446f8edd864a3fac0a53449539b259', 'request_id': 'req-549ef513-3291-440e-b8b8-62cac5876382'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:43.388615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:43.403423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61256b7-4d99-4fa3-b2f7-d88404529859 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:43.407150 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] The token cache is being invalidate because project d0446f8edd864a3fac0a53449539b259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:43.407517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:43.408333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-549ef513-3291-440e-b8b8-62cac5876382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:43.410777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1330/2655] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:43 2022] DELETE /identity/v3/projects/d0446f8edd864a3fac0a53449539b259 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:43.415498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05e77-bb4d-4a73-8ddd-689f22e786b0 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:43.415759 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05e77-bb4d-4a73-8ddd-689f22e786b0 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:43.416036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05e77-bb4d-4a73-8ddd-689f22e786b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:43.442083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f4c05e77-bb4d-4a73-8ddd-689f22e786b0 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:43.443869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f4c05e77-bb4d-4a73-8ddd-689f22e786b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:43.459654 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a0e10-64a9-4458-867e-c5a9698404f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:43.460300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a0e10-64a9-4458-867e-c5a9698404f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:43.460587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a0e10-64a9-4458-867e-c5a9698404f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:43.460891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a0e10-64a9-4458-867e-c5a9698404f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:43.463694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f20a0e10-64a9-4458-867e-c5a9698404f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:43.521309 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1331/2656] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:21:43.530238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa58a9-80f7-4511-98e5-55e98da9502c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:43.530446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa58a9-80f7-4511-98e5-55e98da9502c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:43.530767 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa58a9-80f7-4511-98e5-55e98da9502c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:43.545816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a0e10-64a9-4458-867e-c5a9698404f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:43.547750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a0e10-64a9-4458-867e-c5a9698404f6 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:43.554421 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-f7aa58a9-80f7-4511-98e5-55e98da9502c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:43.556257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f7aa58a9-80f7-4511-98e5-55e98da9502c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:43.563731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1326/2657] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:43.567432 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1332/2658] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:21:43.571582 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a0e10-64a9-4458-867e-c5a9698404f6 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:43.592565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a0e10-64a9-4458-867e-c5a9698404f6 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:43.597315 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3db1b-9248-49ba-b7e9-eff6d76f73b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:43.598628 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3db1b-9248-49ba-b7e9-eff6d76f73b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:43.598912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3db1b-9248-49ba-b7e9-eff6d76f73b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:43.599224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3db1b-9248-49ba-b7e9-eff6d76f73b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:43.600005 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3db1b-9248-49ba-b7e9-eff6d76f73b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:21:43.600308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3db1b-9248-49ba-b7e9-eff6d76f73b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:43.602050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3db1b-9248-49ba-b7e9-eff6d76f73b1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:43.621452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1327/2659] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:21:43.622714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa58a9-80f7-4511-98e5-55e98da9502c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:43.633008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa58a9-80f7-4511-98e5-55e98da9502c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:43.638042 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c3052-85e1-47be-a197-9b300ef62442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:43.638714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c3052-85e1-47be-a197-9b300ef62442 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:43.638996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c3052-85e1-47be-a197-9b300ef62442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:43.639277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c3052-85e1-47be-a197-9b300ef62442 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:43.639996 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c3052-85e1-47be-a197-9b300ef62442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:43.642012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c3052-85e1-47be-a197-9b300ef62442 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:43.673852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1333/2660] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:43 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:21:43.675220 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3db1b-9248-49ba-b7e9-eff6d76f73b1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:43.686129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3db1b-9248-49ba-b7e9-eff6d76f73b1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:43.736269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:43.746356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574cea7-88de-45a5-99fe-3e5998e1ad52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:43.747088 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574cea7-88de-45a5-99fe-3e5998e1ad52 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:43.747375 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574cea7-88de-45a5-99fe-3e5998e1ad52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:43.747771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5574cea7-88de-45a5-99fe-3e5998e1ad52 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:43.748478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574cea7-88de-45a5-99fe-3e5998e1ad52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:43.750443 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5574cea7-88de-45a5-99fe-3e5998e1ad52 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:43.772423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1328/2661] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:43 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:43.776745 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c3052-85e1-47be-a197-9b300ef62442 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:43.787888 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c3052-85e1-47be-a197-9b300ef62442 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:43.792884 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f63da-5a94-4487-95b6-744595c24d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:43.793565 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f63da-5a94-4487-95b6-744595c24d4b admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:43.793850 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f63da-5a94-4487-95b6-744595c24d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:43.794161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f63da-5a94-4487-95b6-744595c24d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:43.795027 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f63da-5a94-4487-95b6-744595c24d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:43.796646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f63da-5a94-4487-95b6-744595c24d4b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:43.808914 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1334/2662] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:43.813100 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574cea7-88de-45a5-99fe-3e5998e1ad52 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:43.823827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5574cea7-88de-45a5-99fe-3e5998e1ad52 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:43.829370 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b239b-5b8b-439f-8cb2-40363b7017e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:43.830077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b239b-5b8b-439f-8cb2-40363b7017e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:43.830399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b239b-5b8b-439f-8cb2-40363b7017e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:43.830663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b239b-5b8b-439f-8cb2-40363b7017e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:43.831677 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b239b-5b8b-439f-8cb2-40363b7017e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:43.833459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b239b-5b8b-439f-8cb2-40363b7017e7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:43.844693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1329/2663] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:43.849034 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f63da-5a94-4487-95b6-744595c24d4b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:43.859564 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f63da-5a94-4487-95b6-744595c24d4b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:43.864628 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391cb49-9fcc-4f40-89fe-88693cd82c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:43.865521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391cb49-9fcc-4f40-89fe-88693cd82c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:43.865939 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391cb49-9fcc-4f40-89fe-88693cd82c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:43.866235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391cb49-9fcc-4f40-89fe-88693cd82c55 admin admin] PATH_INFO: `/v3/projects/f1c7e0c32d6e427282d614c8e5d03a2b/users/f86d7fdfe4974d7f9533c92b1054635d/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:43.881356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391cb49-9fcc-4f40-89fe-88693cd82c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1c7e0c32d6e427282d614c8e5d03a2b, user_id=f86d7fdfe4974d7f9533c92b1054635d, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:43.884116 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391cb49-9fcc-4f40-89fe-88693cd82c55 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:43.899147 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1335/2664] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:43 2022] PUT /identity/v3/projects/f1c7e0c32d6e427282d614c8e5d03a2b/users/f86d7fdfe4974d7f9533c92b1054635d/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:43.909840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3f41c-3f3c-4af6-99c7-cd8b8eccebdb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:43.910221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3f41c-3f3c-4af6-99c7-cd8b8eccebdb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:43.910447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3f41c-3f3c-4af6-99c7-cd8b8eccebdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:43.939700 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-54b3f41c-3f3c-4af6-99c7-cd8b8eccebdb None None] MFA Rules not processed for user `f86d7fdfe4974d7f9533c92b1054635d`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:43.941025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-54b3f41c-3f3c-4af6-99c7-cd8b8eccebdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:44.012895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1330/2665] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:44.017490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391cb49-9fcc-4f40-89fe-88693cd82c55 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:44.081050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391cb49-9fcc-4f40-89fe-88693cd82c55 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:44.085952 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b8708-63ff-47c3-b172-34f99730a6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:44.086619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b8708-63ff-47c3-b172-34f99730a6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:44.086904 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b8708-63ff-47c3-b172-34f99730a6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:44.087181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b8708-63ff-47c3-b172-34f99730a6e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:44.087890 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b8708-63ff-47c3-b172-34f99730a6e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:44.089824 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b8708-63ff-47c3-b172-34f99730a6e8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:44.117262 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1336/2666] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:44 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:21:44.121501 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3f41c-3f3c-4af6-99c7-cd8b8eccebdb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:44.133509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3f41c-3f3c-4af6-99c7-cd8b8eccebdb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:44.193000 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a2b4a-a450-4e5b-a7d1-409afb6c33ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:44.193710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a2b4a-a450-4e5b-a7d1-409afb6c33ed admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:44.194020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a2b4a-a450-4e5b-a7d1-409afb6c33ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:44.194293 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a2b4a-a450-4e5b-a7d1-409afb6c33ed admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:44.195032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a2b4a-a450-4e5b-a7d1-409afb6c33ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:44.196990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a2b4a-a450-4e5b-a7d1-409afb6c33ed admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:44.217484 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1331/2667] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:44 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:44.221517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b8708-63ff-47c3-b172-34f99730a6e8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:44.232754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b8708-63ff-47c3-b172-34f99730a6e8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:44.237723 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e631d-8f6d-48be-81ef-6b47561a0952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:44.238547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e631d-8f6d-48be-81ef-6b47561a0952 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:44.238837 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e631d-8f6d-48be-81ef-6b47561a0952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:44.239113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e631d-8f6d-48be-81ef-6b47561a0952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:44.240055 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e631d-8f6d-48be-81ef-6b47561a0952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:44.241473 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e631d-8f6d-48be-81ef-6b47561a0952 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:44.246986 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:44.254507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1337/2668] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:44.258879 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a2b4a-a450-4e5b-a7d1-409afb6c33ed admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:44.270779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a2b4a-a450-4e5b-a7d1-409afb6c33ed admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:44.275763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae749-f5ab-4040-8c75-64b2f311981d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:44.276669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae749-f5ab-4040-8c75-64b2f311981d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:44.276947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae749-f5ab-4040-8c75-64b2f311981d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:44.277222 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae749-f5ab-4040-8c75-64b2f311981d admin admin] PATH_INFO: `/v3/projects/ffd713a40b2c4007909b24fe2bf2a6a7/users/21f063a0680746aa986b3f0af24cb37d/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:44.294239 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ae749-f5ab-4040-8c75-64b2f311981d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd713a40b2c4007909b24fe2bf2a6a7, user_id=21f063a0680746aa986b3f0af24cb37d, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:44.297122 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ae749-f5ab-4040-8c75-64b2f311981d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:44.315155 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1332/2669] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:44 2022] PUT /identity/v3/projects/ffd713a40b2c4007909b24fe2bf2a6a7/users/21f063a0680746aa986b3f0af24cb37d/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:44.319142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e631d-8f6d-48be-81ef-6b47561a0952 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:44.329816 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e631d-8f6d-48be-81ef-6b47561a0952 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:44.390288 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb754ef-479f-4bff-abf3-f2722dd66224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:44.391111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb754ef-479f-4bff-abf3-f2722dd66224 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:44.391396 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb754ef-479f-4bff-abf3-f2722dd66224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:44.391710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb754ef-479f-4bff-abf3-f2722dd66224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:44.392388 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb754ef-479f-4bff-abf3-f2722dd66224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:44.394471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb754ef-479f-4bff-abf3-f2722dd66224 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:44.405574 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1338/2670] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:21:44.409584 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae749-f5ab-4040-8c75-64b2f311981d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:44.420403 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae749-f5ab-4040-8c75-64b2f311981d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:44.424812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f7ff3-0659-44e8-9dc1-24b2b1328349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:44.425807 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f7ff3-0659-44e8-9dc1-24b2b1328349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:44.426116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f7ff3-0659-44e8-9dc1-24b2b1328349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:44.426390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f7ff3-0659-44e8-9dc1-24b2b1328349 admin admin] PATH_INFO: `/v3/projects/ffd713a40b2c4007909b24fe2bf2a6a7/users/21f063a0680746aa986b3f0af24cb37d/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:44.428450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f7ff3-0659-44e8-9dc1-24b2b1328349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd713a40b2c4007909b24fe2bf2a6a7, user_id=21f063a0680746aa986b3f0af24cb37d, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:44.431214 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f7ff3-0659-44e8-9dc1-24b2b1328349 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:44.449985 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1333/2671] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:44 2022] PUT /identity/v3/projects/ffd713a40b2c4007909b24fe2bf2a6a7/users/21f063a0680746aa986b3f0af24cb37d/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:21:44.519212 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593b3af-453a-467c-900d-52887cf61411 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:44.519503 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593b3af-453a-467c-900d-52887cf61411 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:44.519785 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593b3af-453a-467c-900d-52887cf61411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:44.551054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0593b3af-453a-467c-900d-52887cf61411 None None] MFA Rules not processed for user `21f063a0680746aa986b3f0af24cb37d`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:44.552797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0593b3af-453a-467c-900d-52887cf61411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:44.655858 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1339/2672] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:44.664641 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f7ff3-0659-44e8-9dc1-24b2b1328349 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:44.667117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1e3f7ff3-0659-44e8-9dc1-24b2b1328349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:44.695630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:44.755446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f7ff3-0659-44e8-9dc1-24b2b1328349 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:44.760083 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8f79c-c264-46bd-9075-42ccaa5cef73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:44.760801 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8f79c-c264-46bd-9075-42ccaa5cef73 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:44.761189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8f79c-c264-46bd-9075-42ccaa5cef73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:44.761571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8f79c-c264-46bd-9075-42ccaa5cef73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:44.857105 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8f79c-c264-46bd-9075-42ccaa5cef73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:44.859317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8f79c-c264-46bd-9075-42ccaa5cef73 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:44.871030 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1334/2673] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:45.277129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d361302-393a-442b-8628-0fe4efa25904 None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:45.277321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d361302-393a-442b-8628-0fe4efa25904 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:45.277594 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d361302-393a-442b-8628-0fe4efa25904 None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:45.279325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1340/2674] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:21:45.287663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4b200-776c-432d-ba5a-50f025ea2e25 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:45.287975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4b200-776c-432d-ba5a-50f025ea2e25 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:45.288255 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4b200-776c-432d-ba5a-50f025ea2e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:45.316667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c0d4b200-776c-432d-ba5a-50f025ea2e25 None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:45.317487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c0d4b200-776c-432d-ba5a-50f025ea2e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:45.402967 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1335/2675] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:21:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:45.415518 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d361302-393a-442b-8628-0fe4efa25904 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:45.417491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2d361302-393a-442b-8628-0fe4efa25904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:45.486592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d361302-393a-442b-8628-0fe4efa25904 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:45.491122 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9dbba-a9a8-4091-9de2-7fb92152f88d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a90e384504e8d80264d6bf20f1d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:45.492056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9dbba-a9a8-4091-9de2-7fb92152f88d service glance] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:45.492342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9dbba-a9a8-4091-9de2-7fb92152f88d service glance] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:45.492625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9dbba-a9a8-4091-9de2-7fb92152f88d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:45.504429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9dbba-a9a8-4091-9de2-7fb92152f88d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:45.506945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9dbba-a9a8-4091-9de2-7fb92152f88d service glance] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:45.522547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1341/2676] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:45.595196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60ad9e-8162-4b32-8f31-6c87bd0e34c3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:45.595783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60ad9e-8162-4b32-8f31-6c87bd0e34c3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:45.595783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60ad9e-8162-4b32-8f31-6c87bd0e34c3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:45.597454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1336/2677] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:21:45.603629 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8022d-7769-4326-831d-58276099bdfe None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:45.604006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8022d-7769-4326-831d-58276099bdfe None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:45.604345 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8022d-7769-4326-831d-58276099bdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:45.625746 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c2a8022d-7769-4326-831d-58276099bdfe None None] MFA Rules not processed for user `98649aa38f764af0b41ff5594b94a989`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:45.626988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c2a8022d-7769-4326-831d-58276099bdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:45.647459 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:45.684220 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1342/2678] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:21:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:45.696173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60ad9e-8162-4b32-8f31-6c87bd0e34c3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:45.697969 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0e60ad9e-8162-4b32-8f31-6c87bd0e34c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:45.711523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60ad9e-8162-4b32-8f31-6c87bd0e34c3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:45.715016 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6977fa-f20d-480f-b6fd-35e628ab3351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:45.715667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6977fa-f20d-480f-b6fd-35e628ab3351 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:45.715951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6977fa-f20d-480f-b6fd-35e628ab3351 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:45.716225 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6977fa-f20d-480f-b6fd-35e628ab3351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:45.724162 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6977fa-f20d-480f-b6fd-35e628ab3351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:45.725611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6977fa-f20d-480f-b6fd-35e628ab3351 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:45.734135 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1337/2679] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:45.826166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d706a87-6715-4c85-a564-81010f22f8ac None None] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:45.826452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d706a87-6715-4c85-a564-81010f22f8ac None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:45.826740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d706a87-6715-4c85-a564-81010f22f8ac None None] PATH_INFO: `/` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:45.828360 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1343/2680] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:21:45.836187 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafa15f-21ae-49c3-a9ca-db560ba70ffa None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:45.836451 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafa15f-21ae-49c3-a9ca-db560ba70ffa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:45.836735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafa15f-21ae-49c3-a9ca-db560ba70ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:45.857464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4fafa15f-21ae-49c3-a9ca-db560ba70ffa None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:45.858779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4fafa15f-21ae-49c3-a9ca-db560ba70ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:45.870388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1338/2681] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:21:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:21:45.881857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d706a87-6715-4c85-a564-81010f22f8ac None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:45.883918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-3d706a87-6715-4c85-a564-81010f22f8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:45.900544 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d706a87-6715-4c85-a564-81010f22f8ac None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:45.904671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bccd79-0060-4504-bb08-29cebe876cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:45.905363 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bccd79-0060-4504-bb08-29cebe876cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:45.905647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bccd79-0060-4504-bb08-29cebe876cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:45.906076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bccd79-0060-4504-bb08-29cebe876cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:45.917812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bccd79-0060-4504-bb08-29cebe876cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:45.919334 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bccd79-0060-4504-bb08-29cebe876cf8 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:45.931117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1344/2682] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:46.953459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafa15f-21ae-49c3-a9ca-db560ba70ffa None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:46.955056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4fafa15f-21ae-49c3-a9ca-db560ba70ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:46.969609 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafa15f-21ae-49c3-a9ca-db560ba70ffa None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:46.975539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fb6d0-1508-4e13-9421-a363cdd2a407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:46.976281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fb6d0-1508-4e13-9421-a363cdd2a407 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:46.976618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fb6d0-1508-4e13-9421-a363cdd2a407 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:46.976919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fb6d0-1508-4e13-9421-a363cdd2a407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:46.978908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-205fb6d0-1508-4e13-9421-a363cdd2a407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:46.995574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fb6d0-1508-4e13-9421-a363cdd2a407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:46.997685 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fb6d0-1508-4e13-9421-a363cdd2a407 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:47.010069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1339/2683] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:48.545148 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bccd79-0060-4504-bb08-29cebe876cf8 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:48.554580 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bccd79-0060-4504-bb08-29cebe876cf8 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:48.558484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac80815-1057-4573-80d8-3f5efa2e0965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:48.559325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac80815-1057-4573-80d8-3f5efa2e0965 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:48.559675 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac80815-1057-4573-80d8-3f5efa2e0965 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:48.560015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac80815-1057-4573-80d8-3f5efa2e0965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:48.562107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8ac80815-1057-4573-80d8-3f5efa2e0965 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:48.630267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac80815-1057-4573-80d8-3f5efa2e0965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:48.631734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac80815-1057-4573-80d8-3f5efa2e0965 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:48.641709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1345/2684] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:48.916651 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672a02-513c-4bb0-a900-0b52eae29fd1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:48.917193 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672a02-513c-4bb0-a900-0b52eae29fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:48.917193 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672a02-513c-4bb0-a900-0b52eae29fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:48.937513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-43672a02-513c-4bb0-a900-0b52eae29fd1 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:48.938782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-43672a02-513c-4bb0-a900-0b52eae29fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:48.960709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:48.999176 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1340/2685] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:21:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:49.022354 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac80815-1057-4573-80d8-3f5efa2e0965 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:49.023870 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8ac80815-1057-4573-80d8-3f5efa2e0965 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:49.098469 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac80815-1057-4573-80d8-3f5efa2e0965 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:49.103971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243991-2a7a-4d95-8cbe-33e49a9c8d59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:49.104636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243991-2a7a-4d95-8cbe-33e49a9c8d59 service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:49.104967 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243991-2a7a-4d95-8cbe-33e49a9c8d59 service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:49.105245 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f243991-2a7a-4d95-8cbe-33e49a9c8d59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:49.114863 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f243991-2a7a-4d95-8cbe-33e49a9c8d59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:49.116685 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f243991-2a7a-4d95-8cbe-33e49a9c8d59 service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:49.126233 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1346/2686] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:21:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:21:49.362091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43672a02-513c-4bb0-a900-0b52eae29fd1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:49.376953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43672a02-513c-4bb0-a900-0b52eae29fd1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:49.382505 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f5268-6179-4520-9477-a46a83e8504c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:49.383759 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f5268-6179-4520-9477-a46a83e8504c service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:49.384257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f5268-6179-4520-9477-a46a83e8504c service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:49.384778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f5268-6179-4520-9477-a46a83e8504c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:49.397434 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f5268-6179-4520-9477-a46a83e8504c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:49.399145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f5268-6179-4520-9477-a46a83e8504c service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:49.412978 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1341/2687] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:53.204877 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243991-2a7a-4d95-8cbe-33e49a9c8d59 service swift] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:53.206491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-7f243991-2a7a-4d95-8cbe-33e49a9c8d59 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:53.258416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=100 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:53.299651 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f243991-2a7a-4d95-8cbe-33e49a9c8d59 service swift] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:53.313810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f99b9-8360-4b82-8dff-cd949cf55ad1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:53.315880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f99b9-8360-4b82-8dff-cd949cf55ad1 service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:53.315880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f99b9-8360-4b82-8dff-cd949cf55ad1 service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:53.315880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f99b9-8360-4b82-8dff-cd949cf55ad1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:53.336675 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-872f99b9-8360-4b82-8dff-cd949cf55ad1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:53.368270 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f99b9-8360-4b82-8dff-cd949cf55ad1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:53.370165 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f99b9-8360-4b82-8dff-cd949cf55ad1 service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:53.394226 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1347/2688] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:54.756944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f5268-6179-4520-9477-a46a83e8504c service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:54.769868 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f5268-6179-4520-9477-a46a83e8504c service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:54.790031 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:54.791492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:54.792139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:54.795748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] PATH_INFO: `/v3/users/f86d7fdfe4974d7f9533c92b1054635d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:54.798132 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86d7fdfe4974d7f9533c92b1054635d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:54.800334 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:54.908607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86d7fdfe4974d7f9533c92b1054635d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:54.909778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86d7fdfe4974d7f9533c92b1054635d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:54.911264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86d7fdfe4974d7f9533c92b1054635d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:54.925304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86d7fdfe4974d7f9533c92b1054635d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:54.940408 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86d7fdfe4974d7f9533c92b1054635d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:54.968859 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1342/2689] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:54 2022] DELETE /identity/v3/users/f86d7fdfe4974d7f9533c92b1054635d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:54.970908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ca76-5e78-4ae7-a76a-9bbf5ccf5d1a None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:54.971390 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ca76-5e78-4ae7-a76a-9bbf5ccf5d1a None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:54.971478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ca76-5e78-4ae7-a76a-9bbf5ccf5d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:55.019978 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d6b1ca76-5e78-4ae7-a76a-9bbf5ccf5d1a None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:55.021900 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d6b1ca76-5e78-4ae7-a76a-9bbf5ccf5d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:55.147047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1348/2690] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:55.159884 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:55.161587 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:55.274713 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08e8c7-a5d3-4ffb-9a73-f7bb72bd37c7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:55.278393 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9601bb-f1c4-43fe-9184-fb3d802d2529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:55.279061 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9601bb-f1c4-43fe-9184-fb3d802d2529 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:55.279344 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9601bb-f1c4-43fe-9184-fb3d802d2529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:55.279621 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9601bb-f1c4-43fe-9184-fb3d802d2529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:55.290592 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9601bb-f1c4-43fe-9184-fb3d802d2529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:55.292066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9601bb-f1c4-43fe-9184-fb3d802d2529 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:55.315841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1343/2691] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:56.124092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1ca76-5e78-4ae7-a76a-9bbf5ccf5d1a None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:56.132536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d6b1ca76-5e78-4ae7-a76a-9bbf5ccf5d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:56.164160 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1ca76-5e78-4ae7-a76a-9bbf5ccf5d1a None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:56.172068 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:56.174161 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:56.174887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:56.175377 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] PATH_INFO: `/v3/projects/f1c7e0c32d6e427282d614c8e5d03a2b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:56.179184 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1c7e0c32d6e427282d614c8e5d03a2b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:56.183321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:56.302379 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1c7e0c32d6e427282d614c8e5d03a2b', 'request_id': 'req-d6a7e15c-1bd9-43e7-858f-9da6eb530798'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:56.313950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1c7e0c32d6e427282d614c8e5d03a2b', 'request_id': 'req-d6a7e15c-1bd9-43e7-858f-9da6eb530798'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:56.323496 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1c7e0c32d6e427282d614c8e5d03a2b', 'request_id': 'req-d6a7e15c-1bd9-43e7-858f-9da6eb530798'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:56.336588 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] The token cache is being invalidate because project f1c7e0c32d6e427282d614c8e5d03a2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:56.336930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:56.337696 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:56.342972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1349/2692] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:56 2022] DELETE /identity/v3/projects/f1c7e0c32d6e427282d614c8e5d03a2b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:56.344710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9601bb-f1c4-43fe-9184-fb3d802d2529 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:56.359768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9601bb-f1c4-43fe-9184-fb3d802d2529 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:56.410529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:56.452731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c73c3-1698-44ac-a1ee-b1488f3978d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:56.454663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c73c3-1698-44ac-a1ee-b1488f3978d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:56.454954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c73c3-1698-44ac-a1ee-b1488f3978d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:56.455378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c73c3-1698-44ac-a1ee-b1488f3978d6 admin admin] PATH_INFO: `/v3/users/21f063a0680746aa986b3f0af24cb37d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:56.457157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c73c3-1698-44ac-a1ee-b1488f3978d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21f063a0680746aa986b3f0af24cb37d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:56.460304 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c73c3-1698-44ac-a1ee-b1488f3978d6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:56.545180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cd6c73c3-1698-44ac-a1ee-b1488f3978d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21f063a0680746aa986b3f0af24cb37d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:56.546082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cd6c73c3-1698-44ac-a1ee-b1488f3978d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21f063a0680746aa986b3f0af24cb37d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:56.546684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cd6c73c3-1698-44ac-a1ee-b1488f3978d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21f063a0680746aa986b3f0af24cb37d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:56.565112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cd6c73c3-1698-44ac-a1ee-b1488f3978d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21f063a0680746aa986b3f0af24cb37d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:56.586028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cd6c73c3-1698-44ac-a1ee-b1488f3978d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21f063a0680746aa986b3f0af24cb37d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:56.631947 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1344/2693] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:56 2022] DELETE /identity/v3/users/21f063a0680746aa986b3f0af24cb37d => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:57.183266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:57.186943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:57.210724 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7e15c-1bd9-43e7-858f-9da6eb530798 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:57.301351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:57.313987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:57.315668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:57.316176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:57.317280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] PATH_INFO: `/v3/projects/ffd713a40b2c4007909b24fe2bf2a6a7` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:57.319531 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd713a40b2c4007909b24fe2bf2a6a7)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:57.322072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:57.388878 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd713a40b2c4007909b24fe2bf2a6a7', 'request_id': 'req-7452e30d-ba68-4b96-8f26-65c788bfafc3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:57.398793 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd713a40b2c4007909b24fe2bf2a6a7', 'request_id': 'req-7452e30d-ba68-4b96-8f26-65c788bfafc3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:57.405373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd713a40b2c4007909b24fe2bf2a6a7', 'request_id': 'req-7452e30d-ba68-4b96-8f26-65c788bfafc3'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:57.416797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] The token cache is being invalidate because project ffd713a40b2c4007909b24fe2bf2a6a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:57.417163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:57.417789 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-7452e30d-ba68-4b96-8f26-65c788bfafc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:57.419488 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1350/2694] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:57 2022] DELETE /identity/v3/projects/ffd713a40b2c4007909b24fe2bf2a6a7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:57.426208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1678fe-12c8-499b-8969-d1daeafa70d4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:57.427085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1678fe-12c8-499b-8969-d1daeafa70d4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:57.427363 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1678fe-12c8-499b-8969-d1daeafa70d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:57.449756 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ec1678fe-12c8-499b-8969-d1daeafa70d4 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:57.451002 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ec1678fe-12c8-499b-8969-d1daeafa70d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:57.520251 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1345/2695] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:57.531552 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b413b-d10d-4f8c-943a-bad538f7be9f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:57.531893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b413b-d10d-4f8c-943a-bad538f7be9f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:57.532156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b413b-d10d-4f8c-943a-bad538f7be9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:57.552943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-695b413b-d10d-4f8c-943a-bad538f7be9f None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:57.554381 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-695b413b-d10d-4f8c-943a-bad538f7be9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:57.562479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1351/2696] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:21:57.567155 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1678fe-12c8-499b-8969-d1daeafa70d4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:57.577052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1678fe-12c8-499b-8969-d1daeafa70d4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:57.583196 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6bbd5-69ac-4cfa-a6a8-60d3faec27b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:57.584209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6bbd5-69ac-4cfa-a6a8-60d3faec27b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:57.584538 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6bbd5-69ac-4cfa-a6a8-60d3faec27b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:57.584818 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6bbd5-69ac-4cfa-a6a8-60d3faec27b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:57.585563 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6bbd5-69ac-4cfa-a6a8-60d3faec27b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:21:57.586087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6bbd5-69ac-4cfa-a6a8-60d3faec27b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:57.587707 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6bbd5-69ac-4cfa-a6a8-60d3faec27b9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:57.600445 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1346/2697] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:21:57.606751 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b413b-d10d-4f8c-943a-bad538f7be9f None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:57.616301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b413b-d10d-4f8c-943a-bad538f7be9f None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:57.621023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767d635-3444-4277-b4ec-8add268377a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:57.621760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767d635-3444-4277-b4ec-8add268377a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:57.622134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767d635-3444-4277-b4ec-8add268377a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:57.622484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767d635-3444-4277-b4ec-8add268377a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:57.623279 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767d635-3444-4277-b4ec-8add268377a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:57.625308 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767d635-3444-4277-b4ec-8add268377a9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:57.653085 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1352/2698] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:21:57 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:57.657453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6bbd5-69ac-4cfa-a6a8-60d3faec27b9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:57.667992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6bbd5-69ac-4cfa-a6a8-60d3faec27b9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:57.707469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:57.729705 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab15169-6214-4529-b248-3c5d311b2661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:57.730420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab15169-6214-4529-b248-3c5d311b2661 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:57.730709 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab15169-6214-4529-b248-3c5d311b2661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:57.730989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab15169-6214-4529-b248-3c5d311b2661 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:57.731683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab15169-6214-4529-b248-3c5d311b2661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:57.733611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab15169-6214-4529-b248-3c5d311b2661 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:57.760783 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1347/2699] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:21:57 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:21:57.762397 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767d635-3444-4277-b4ec-8add268377a9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:57.777737 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767d635-3444-4277-b4ec-8add268377a9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:57.779131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab15169-6214-4529-b248-3c5d311b2661 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:57.781794 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7ab15169-6214-4529-b248-3c5d311b2661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:57.790015 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd271c-7fa7-4f0e-95e2-22de9e4fff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:57.790921 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd271c-7fa7-4f0e-95e2-22de9e4fff28 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:57.794674 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd271c-7fa7-4f0e-95e2-22de9e4fff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:57.796103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd271c-7fa7-4f0e-95e2-22de9e4fff28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:57.796103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd271c-7fa7-4f0e-95e2-22de9e4fff28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:57.797458 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd271c-7fa7-4f0e-95e2-22de9e4fff28 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:57.806035 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab15169-6214-4529-b248-3c5d311b2661 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:57.809923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1353/2700] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:21:57.815856 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd271c-7fa7-4f0e-95e2-22de9e4fff28 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:57.820452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:57.827941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:57.828264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:57.828552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] PATH_INFO: `/v3/users/5ba79c77fcd54bbcb673a5cba1ed2975` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:57.828677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd271c-7fa7-4f0e-95e2-22de9e4fff28 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:57.829936 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba79c77fcd54bbcb673a5cba1ed2975)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:57.832113 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:57.836593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33f3e6-c893-439d-b2ea-7aacc4204803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:57.837276 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33f3e6-c893-439d-b2ea-7aacc4204803 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:57.837753 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33f3e6-c893-439d-b2ea-7aacc4204803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:57.838062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33f3e6-c893-439d-b2ea-7aacc4204803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:57.838857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33f3e6-c893-439d-b2ea-7aacc4204803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:57.841231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33f3e6-c893-439d-b2ea-7aacc4204803 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:57.856291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1354/2701] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:21:57.857403 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33f3e6-c893-439d-b2ea-7aacc4204803 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:57.871717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33f3e6-c893-439d-b2ea-7aacc4204803 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:57.878551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd51a3-8982-42ed-a11a-54adf6396bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:57.879594 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd51a3-8982-42ed-a11a-54adf6396bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:57.879934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd51a3-8982-42ed-a11a-54adf6396bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:57.880217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd51a3-8982-42ed-a11a-54adf6396bfd admin admin] PATH_INFO: `/v3/projects/f6f374944803464a9fd851b28092e6a6/users/4e86adf4115b43518183fc3d984cbc60/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:57.890414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba79c77fcd54bbcb673a5cba1ed2975'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:57.891249 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba79c77fcd54bbcb673a5cba1ed2975'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:57.891885 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba79c77fcd54bbcb673a5cba1ed2975'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:57.896452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd51a3-8982-42ed-a11a-54adf6396bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6f374944803464a9fd851b28092e6a6, user_id=4e86adf4115b43518183fc3d984cbc60, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:57.899415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd51a3-8982-42ed-a11a-54adf6396bfd admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:57.901071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba79c77fcd54bbcb673a5cba1ed2975'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:57.909086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba79c77fcd54bbcb673a5cba1ed2975'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:57.914474 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1355/2702] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:21:57 2022] PUT /identity/v3/projects/f6f374944803464a9fd851b28092e6a6/users/4e86adf4115b43518183fc3d984cbc60/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:21:57.927584 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401551d-fff1-494b-b945-3cb62cf78602 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:57.927874 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401551d-fff1-494b-b945-3cb62cf78602 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:57.928156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401551d-fff1-494b-b945-3cb62cf78602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:57.929574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1348/2703] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:57 2022] DELETE /identity/v3/users/5ba79c77fcd54bbcb673a5cba1ed2975 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:21:57.955211 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4401551d-fff1-494b-b945-3cb62cf78602 None None] MFA Rules not processed for user `4e86adf4115b43518183fc3d984cbc60`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:21:57.956443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4401551d-fff1-494b-b945-3cb62cf78602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:58.028857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1356/2704] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:21:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:21:58.041360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:58.043062 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:58.113043 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7252cb-f3c6-413f-853b-522beaadf022 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:58.118090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79383c6f-b37f-44fd-84b2-edb45b8b80ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:58.118740 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401551d-fff1-494b-b945-3cb62cf78602 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:58.118897 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79383c6f-b37f-44fd-84b2-edb45b8b80ab service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:58.119264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79383c6f-b37f-44fd-84b2-edb45b8b80ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:58.119586 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79383c6f-b37f-44fd-84b2-edb45b8b80ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:58.120550 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4401551d-fff1-494b-b945-3cb62cf78602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:58.132805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:58.143767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79383c6f-b37f-44fd-84b2-edb45b8b80ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:58.145360 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79383c6f-b37f-44fd-84b2-edb45b8b80ab service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:58.159741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1349/2705] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:21:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:21:58.219478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401551d-fff1-494b-b945-3cb62cf78602 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:58.225260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:58.226269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:58.226551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:58.226825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] PATH_INFO: `/v3/projects/33b3e0ba3aa644ddbbee8f56be21ad06` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:58.228189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33b3e0ba3aa644ddbbee8f56be21ad06)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:58.230176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:58.280384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33b3e0ba3aa644ddbbee8f56be21ad06', 'request_id': 'req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.287452 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33b3e0ba3aa644ddbbee8f56be21ad06', 'request_id': 'req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.294129 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33b3e0ba3aa644ddbbee8f56be21ad06', 'request_id': 'req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.305413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] The token cache is being invalidate because project 33b3e0ba3aa644ddbbee8f56be21ad06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:58.305670 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.306499 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.308217 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1357/2706] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:58 2022] DELETE /identity/v3/projects/33b3e0ba3aa644ddbbee8f56be21ad06 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:21:58.312351 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79383c6f-b37f-44fd-84b2-edb45b8b80ab service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:58.325453 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79383c6f-b37f-44fd-84b2-edb45b8b80ab service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:58.371573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:58.391504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af17362-0037-4d98-ae2d-45ab7c9ab6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:58.392491 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af17362-0037-4d98-ae2d-45ab7c9ab6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:58.392849 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af17362-0037-4d98-ae2d-45ab7c9ab6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:58.393444 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af17362-0037-4d98-ae2d-45ab7c9ab6c9 admin admin] PATH_INFO: `/v3/users/b89afc9411be4575b1b94d3c1e7b1eb2` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:58.395102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af17362-0037-4d98-ae2d-45ab7c9ab6c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b89afc9411be4575b1b94d3c1e7b1eb2)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:58.397346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af17362-0037-4d98-ae2d-45ab7c9ab6c9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:58.449590 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2af17362-0037-4d98-ae2d-45ab7c9ab6c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b89afc9411be4575b1b94d3c1e7b1eb2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.450342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2af17362-0037-4d98-ae2d-45ab7c9ab6c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b89afc9411be4575b1b94d3c1e7b1eb2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.451070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2af17362-0037-4d98-ae2d-45ab7c9ab6c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b89afc9411be4575b1b94d3c1e7b1eb2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.461062 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2af17362-0037-4d98-ae2d-45ab7c9ab6c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b89afc9411be4575b1b94d3c1e7b1eb2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.472976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2af17362-0037-4d98-ae2d-45ab7c9ab6c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b89afc9411be4575b1b94d3c1e7b1eb2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.493291 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1350/2707] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:21:58 2022] DELETE /identity/v3/users/b89afc9411be4575b1b94d3c1e7b1eb2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:21:58.679450 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:21:58.680893 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:21:58.694257 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9e1a0-23aa-4d67-bddc-45f2c2efb7f8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:21:58.757663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:21:58.758560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:21:58.758840 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:21:58.759095 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] PATH_INFO: `/v3/projects/abfe2a72af8d4f658c560f45f82d0421` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:21:58.760306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=abfe2a72af8d4f658c560f45f82d0421)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:21:58.762240 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:21:58.804191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:21:58.816067 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abfe2a72af8d4f658c560f45f82d0421', 'request_id': 'req-10177054-caa8-459f-9dc5-3e99d24f0ceb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.824312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abfe2a72af8d4f658c560f45f82d0421', 'request_id': 'req-10177054-caa8-459f-9dc5-3e99d24f0ceb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.831431 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abfe2a72af8d4f658c560f45f82d0421', 'request_id': 'req-10177054-caa8-459f-9dc5-3e99d24f0ceb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.844639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] The token cache is being invalidate because project abfe2a72af8d4f658c560f45f82d0421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:21:58.844949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.845694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:21:58.847432 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1358/2708] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:21:58 2022] DELETE /identity/v3/projects/abfe2a72af8d4f658c560f45f82d0421 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:22:08.489669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:08.840748 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:09.926323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456c16a-fef0-46a8-9b84-d005b2e2cdf8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:09.926854 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456c16a-fef0-46a8-9b84-d005b2e2cdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:09.926854 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456c16a-fef0-46a8-9b84-d005b2e2cdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:09.946632 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f456c16a-fef0-46a8-9b84-d005b2e2cdf8 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:22:09.947831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f456c16a-fef0-46a8-9b84-d005b2e2cdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:10.005605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1351/2709] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 4 09:22:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:22:10.019176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:10.020905 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:10.097497 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10177054-caa8-459f-9dc5-3e99d24f0ceb admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:10.106186 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5639028-5e6e-4231-9cbf-8a257badc858 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:10.107132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5639028-5e6e-4231-9cbf-8a257badc858 service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:10.107522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5639028-5e6e-4231-9cbf-8a257badc858 service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:10.107898 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5639028-5e6e-4231-9cbf-8a257badc858 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:10.117711 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5639028-5e6e-4231-9cbf-8a257badc858 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:10.119393 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5639028-5e6e-4231-9cbf-8a257badc858 service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:10.128339 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1359/2710] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:22:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:22:17.953238 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456c16a-fef0-46a8-9b84-d005b2e2cdf8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:17.954662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f456c16a-fef0-46a8-9b84-d005b2e2cdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:18.026943 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456c16a-fef0-46a8-9b84-d005b2e2cdf8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:18.031782 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:18.032731 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:18.033011 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:18.033294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] PATH_INFO: `/v3/users/4e86adf4115b43518183fc3d984cbc60` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:18.034565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e86adf4115b43518183fc3d984cbc60)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:18.036503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:18.093134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e86adf4115b43518183fc3d984cbc60'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:18.093793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e86adf4115b43518183fc3d984cbc60'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:18.094424 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e86adf4115b43518183fc3d984cbc60'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:18.103664 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e86adf4115b43518183fc3d984cbc60'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:18.111111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e86adf4115b43518183fc3d984cbc60'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:18.128497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1352/2711] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:22:17 2022] DELETE /identity/v3/users/4e86adf4115b43518183fc3d984cbc60 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:22:18.134016 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d4060-203e-4695-ade4-becd94e80d46 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:18.134514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d4060-203e-4695-ade4-becd94e80d46 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:18.134613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d4060-203e-4695-ade4-becd94e80d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:18.157463 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-db5d4060-203e-4695-ade4-becd94e80d46 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:22:18.159325 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-db5d4060-203e-4695-ade4-becd94e80d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:18.222467 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1360/2712] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:22:18.232546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:18.234056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:18.266871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:18.304492 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940d6ff-d0d3-481e-b181-b74bc6846764 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:18.308517 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc00da-2f32-474f-ae76-b156d8d56a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:18.309166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc00da-2f32-474f-ae76-b156d8d56a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:18.309454 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc00da-2f32-474f-ae76-b156d8d56a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:18.309726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc00da-2f32-474f-ae76-b156d8d56a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:18.319473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc00da-2f32-474f-ae76-b156d8d56a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:18.320922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc00da-2f32-474f-ae76-b156d8d56a42 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:18.330528 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1353/2713] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:22:18.852880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d4060-203e-4695-ade4-becd94e80d46 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:18.854420 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-db5d4060-203e-4695-ade4-becd94e80d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:18.867056 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d4060-203e-4695-ade4-becd94e80d46 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:18.873151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:18.874679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:18.875008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:18.875293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] PATH_INFO: `/v3/projects/f6f374944803464a9fd851b28092e6a6` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:18.876817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6f374944803464a9fd851b28092e6a6)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:18.878773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:18.929028 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6f374944803464a9fd851b28092e6a6', 'request_id': 'req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:18.938146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6f374944803464a9fd851b28092e6a6', 'request_id': 'req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:18.944666 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6f374944803464a9fd851b28092e6a6', 'request_id': 'req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:18.955062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] The token cache is being invalidate because project f6f374944803464a9fd851b28092e6a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:22:18.955347 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:18.955992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f504d89d-08e4-4dc9-998f-dfbeab5dfd19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:18.957637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1361/2714] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:22:18 2022] DELETE /identity/v3/projects/f6f374944803464a9fd851b28092e6a6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:22:18.962948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2deee-ef60-47e0-a3bc-0665503f62f5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:18.963439 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2deee-ef60-47e0-a3bc-0665503f62f5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:18.963537 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2deee-ef60-47e0-a3bc-0665503f62f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:18.981989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a6c2deee-ef60-47e0-a3bc-0665503f62f5 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:22:18.983200 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a6c2deee-ef60-47e0-a3bc-0665503f62f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:19.052879 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1354/2715] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:22:19.060826 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d920ee-1596-45ed-bf49-273630df3d03 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:19.061187 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d920ee-1596-45ed-bf49-273630df3d03 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:19.061465 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d920ee-1596-45ed-bf49-273630df3d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:19.080315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-58d920ee-1596-45ed-bf49-273630df3d03 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:22:19.081521 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-58d920ee-1596-45ed-bf49-273630df3d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:19.089829 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1362/2716] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:22:19.095104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2deee-ef60-47e0-a3bc-0665503f62f5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:19.106878 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2deee-ef60-47e0-a3bc-0665503f62f5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:19.112261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c40f9-c9e3-4a88-9dfd-aad2d0aed9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:19.112917 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c40f9-c9e3-4a88-9dfd-aad2d0aed9fc admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:19.113199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c40f9-c9e3-4a88-9dfd-aad2d0aed9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:19.113472 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c40f9-c9e3-4a88-9dfd-aad2d0aed9fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:19.114240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c40f9-c9e3-4a88-9dfd-aad2d0aed9fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:22:19.114551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c40f9-c9e3-4a88-9dfd-aad2d0aed9fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:19.116221 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c40f9-c9e3-4a88-9dfd-aad2d0aed9fc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:19.130429 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1355/2717] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:22:19.132716 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d920ee-1596-45ed-bf49-273630df3d03 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:19.142810 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d920ee-1596-45ed-bf49-273630df3d03 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:19.148918 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6bc80-e7dc-4470-a1dd-b430f6e905d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:19.149583 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6bc80-e7dc-4470-a1dd-b430f6e905d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:19.149865 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6bc80-e7dc-4470-a1dd-b430f6e905d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:19.150163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6bc80-e7dc-4470-a1dd-b430f6e905d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:19.151006 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6bc80-e7dc-4470-a1dd-b430f6e905d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:19.152909 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6bc80-e7dc-4470-a1dd-b430f6e905d8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:19.156739 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:19.184900 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1363/2718] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:22:19 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:22:19.189193 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c40f9-c9e3-4a88-9dfd-aad2d0aed9fc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:19.199557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c40f9-c9e3-4a88-9dfd-aad2d0aed9fc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:19.261259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aaee92-bb51-46e0-9f5a-e4903c39e954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:19.261949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aaee92-bb51-46e0-9f5a-e4903c39e954 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:19.262243 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aaee92-bb51-46e0-9f5a-e4903c39e954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:19.262523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aaee92-bb51-46e0-9f5a-e4903c39e954 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:19.263216 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aaee92-bb51-46e0-9f5a-e4903c39e954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:19.265172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aaee92-bb51-46e0-9f5a-e4903c39e954 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:19.289038 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1356/2719] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:22:19 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:22:19.293236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6bc80-e7dc-4470-a1dd-b430f6e905d8 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:19.303671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6bc80-e7dc-4470-a1dd-b430f6e905d8 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:19.308351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18eb3e-8074-4798-ada6-78edac00ae2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:19.309005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18eb3e-8074-4798-ada6-78edac00ae2a admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:19.309323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18eb3e-8074-4798-ada6-78edac00ae2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:19.309594 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18eb3e-8074-4798-ada6-78edac00ae2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:19.310350 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18eb3e-8074-4798-ada6-78edac00ae2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:19.311942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18eb3e-8074-4798-ada6-78edac00ae2a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:19.322680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1364/2720] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:22:19.326779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aaee92-bb51-46e0-9f5a-e4903c39e954 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:19.337626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aaee92-bb51-46e0-9f5a-e4903c39e954 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:19.342903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae3963-62b6-4d11-bfab-136a8228ffc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:19.343567 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae3963-62b6-4d11-bfab-136a8228ffc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:19.343854 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae3963-62b6-4d11-bfab-136a8228ffc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:19.344102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae3963-62b6-4d11-bfab-136a8228ffc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:19.344835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae3963-62b6-4d11-bfab-136a8228ffc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:19.346596 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae3963-62b6-4d11-bfab-136a8228ffc2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:19.357040 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1357/2721] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:22:19.361198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18eb3e-8074-4798-ada6-78edac00ae2a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:19.371830 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18eb3e-8074-4798-ada6-78edac00ae2a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:19.377256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d69d68-4fef-4130-a9ee-9f9dd2af05a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:19.378176 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69d68-4fef-4130-a9ee-9f9dd2af05a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:19.378464 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69d68-4fef-4130-a9ee-9f9dd2af05a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:19.378739 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69d68-4fef-4130-a9ee-9f9dd2af05a5 admin admin] PATH_INFO: `/v3/projects/c3c843f29aa34e9a98f79b32652e489e/users/87f6817fa88e465f82993959e170cce9/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:19.396482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d69d68-4fef-4130-a9ee-9f9dd2af05a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c843f29aa34e9a98f79b32652e489e, user_id=87f6817fa88e465f82993959e170cce9, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:19.399532 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d69d68-4fef-4130-a9ee-9f9dd2af05a5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:19.417227 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1365/2722] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:22:19 2022] PUT /identity/v3/projects/c3c843f29aa34e9a98f79b32652e489e/users/87f6817fa88e465f82993959e170cce9/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:22:19.428742 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f26df3-b7e8-4108-9ce0-c242c7fb0d24 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:19.429041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f26df3-b7e8-4108-9ce0-c242c7fb0d24 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:19.429327 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f26df3-b7e8-4108-9ce0-c242c7fb0d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:19.456058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-27f26df3-b7e8-4108-9ce0-c242c7fb0d24 None None] MFA Rules not processed for user `87f6817fa88e465f82993959e170cce9`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:22:19.457250 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-27f26df3-b7e8-4108-9ce0-c242c7fb0d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:19.479467 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:19.525599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1358/2723] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:22:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:22:19.538342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d69d68-4fef-4130-a9ee-9f9dd2af05a5 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:19.541410 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-32d69d68-4fef-4130-a9ee-9f9dd2af05a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:19.615191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d69d68-4fef-4130-a9ee-9f9dd2af05a5 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:19.619498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d4bbe-c4f4-4fb3-845c-3d8cfb7bf774 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a90e384504e8d80264d6bf20f1d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:19.620142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d4bbe-c4f4-4fb3-845c-3d8cfb7bf774 service glance] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:19.620435 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d4bbe-c4f4-4fb3-845c-3d8cfb7bf774 service glance] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:19.620709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d4bbe-c4f4-4fb3-845c-3d8cfb7bf774 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:19.630100 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d4bbe-c4f4-4fb3-845c-3d8cfb7bf774 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:19.631570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d4bbe-c4f4-4fb3-845c-3d8cfb7bf774 service glance] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:19.640534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1366/2724] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:22:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:22:29.523009 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=17 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:29.637080 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:48.416025 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f26df3-b7e8-4108-9ce0-c242c7fb0d24 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:48.418458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-27f26df3-b7e8-4108-9ce0-c242c7fb0d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:48.486011 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f26df3-b7e8-4108-9ce0-c242c7fb0d24 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:48.490007 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f62d89-b56d-443f-a52b-379c899a955d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:48.490733 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f62d89-b56d-443f-a52b-379c899a955d service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:48.491017 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f62d89-b56d-443f-a52b-379c899a955d service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:48.491294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f62d89-b56d-443f-a52b-379c899a955d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:48.493247 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f1f62d89-b56d-443f-a52b-379c899a955d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:48.558907 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f62d89-b56d-443f-a52b-379c899a955d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:48.560392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f62d89-b56d-443f-a52b-379c899a955d service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:48.570158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1359/2725] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:22:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:22:54.266846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d4bbe-c4f4-4fb3-845c-3d8cfb7bf774 service glance] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:54.337784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d4bbe-c4f4-4fb3-845c-3d8cfb7bf774 service glance] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:54.342650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:54.343529 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:54.343835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:54.344112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] PATH_INFO: `/v3/users/87f6817fa88e465f82993959e170cce9` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:54.345321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f6817fa88e465f82993959e170cce9)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:54.347258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:54.402283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f6817fa88e465f82993959e170cce9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:54.403010 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f6817fa88e465f82993959e170cce9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:54.412029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f6817fa88e465f82993959e170cce9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:54.420295 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f6817fa88e465f82993959e170cce9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:54.421030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f6817fa88e465f82993959e170cce9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:54.440680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1367/2726] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:22:54 2022] DELETE /identity/v3/users/87f6817fa88e465f82993959e170cce9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:22:54.445599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331de50f-07d7-45ad-a264-b971c9dbf51f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:54.446182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331de50f-07d7-45ad-a264-b971c9dbf51f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:54.446182 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331de50f-07d7-45ad-a264-b971c9dbf51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:54.464626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-331de50f-07d7-45ad-a264-b971c9dbf51f None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:22:54.465848 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-331de50f-07d7-45ad-a264-b971c9dbf51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:54.527676 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1360/2727] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:22:54.537590 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:54.539592 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:54.600043 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1bc84-cc93-4c2f-85fd-5a9ad1890994 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:54.604522 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c20a4-e5c0-4e2b-8ecd-053406b6cf3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:54.605144 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c20a4-e5c0-4e2b-8ecd-053406b6cf3b service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:54.605476 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c20a4-e5c0-4e2b-8ecd-053406b6cf3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:54.605726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c20a4-e5c0-4e2b-8ecd-053406b6cf3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:54.615563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c20a4-e5c0-4e2b-8ecd-053406b6cf3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:54.617014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c20a4-e5c0-4e2b-8ecd-053406b6cf3b service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:54.627430 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1368/2728] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:22:54.905078 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331de50f-07d7-45ad-a264-b971c9dbf51f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:54.906658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-331de50f-07d7-45ad-a264-b971c9dbf51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:54.919457 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331de50f-07d7-45ad-a264-b971c9dbf51f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:54.924773 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:54.925800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:54.926217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:54.926392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] PATH_INFO: `/v3/projects/c3c843f29aa34e9a98f79b32652e489e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:54.927716 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c843f29aa34e9a98f79b32652e489e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:54.929686 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:54.982531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c843f29aa34e9a98f79b32652e489e', 'request_id': 'req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:54.984213 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:54.991452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c843f29aa34e9a98f79b32652e489e', 'request_id': 'req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:54.998254 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c843f29aa34e9a98f79b32652e489e', 'request_id': 'req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:55.008525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] The token cache is being invalidate because project c3c843f29aa34e9a98f79b32652e489e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:22:55.008817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:55.009496 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:22:55.011242 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1361/2729] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:22:54 2022] DELETE /identity/v3/projects/c3c843f29aa34e9a98f79b32652e489e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:22:56.569126 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909d266-c43c-4e87-b1cb-7136648c6ed6 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:56.569768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909d266-c43c-4e87-b1cb-7136648c6ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:56.569768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909d266-c43c-4e87-b1cb-7136648c6ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:56.588060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2909d266-c43c-4e87-b1cb-7136648c6ed6 None None] MFA Rules not processed for user `f884304cbb764a8ab4927c3ba8db892d`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:22:56.589256 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2909d266-c43c-4e87-b1cb-7136648c6ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:56.607748 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:22:56.649820 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1369/2730] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:22:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:22:58.673754 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:58.675156 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909d266-c43c-4e87-b1cb-7136648c6ed6 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:22:58.675557 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:58.676542 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2909d266-c43c-4e87-b1cb-7136648c6ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:58.746670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aae6e6-e0bd-46a4-b103-2c9ca9f09c02 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:58.747168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909d266-c43c-4e87-b1cb-7136648c6ed6 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:22:58.752347 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721caba-b5d5-4c88-88ae-ae9752026f2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:58.752520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f810d-967d-4122-99fe-094f518eeb10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:22:58.752994 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721caba-b5d5-4c88-88ae-ae9752026f2e service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:58.753107 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f810d-967d-4122-99fe-094f518eeb10 service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:22:58.753275 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721caba-b5d5-4c88-88ae-ae9752026f2e service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:58.753398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f810d-967d-4122-99fe-094f518eeb10 service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:22:58.753531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721caba-b5d5-4c88-88ae-ae9752026f2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:58.753641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f810d-967d-4122-99fe-094f518eeb10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:22:58.755353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6721caba-b5d5-4c88-88ae-ae9752026f2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:58.755480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b58f810d-967d-4122-99fe-094f518eeb10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:22:58.768130 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f810d-967d-4122-99fe-094f518eeb10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:58.768353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721caba-b5d5-4c88-88ae-ae9752026f2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:22:58.769737 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f810d-967d-4122-99fe-094f518eeb10 service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:58.772546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721caba-b5d5-4c88-88ae-ae9752026f2e service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:22:58.781113 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1370/2731] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:22:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:22:58.784857 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1362/2732] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:22:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:23:04.050344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0bf5a-f73f-4dd5-8a7d-cd3789a76b78 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:23:04.050869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0bf5a-f73f-4dd5-8a7d-cd3789a76b78 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:23:04.050982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0bf5a-f73f-4dd5-8a7d-cd3789a76b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:23:04.068860 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a9e0bf5a-f73f-4dd5-8a7d-cd3789a76b78 None None] MFA Rules not processed for user `bcad5b2163e647f386b482731e1de798`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:23:04.070094 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a9e0bf5a-f73f-4dd5-8a7d-cd3789a76b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:23:04.127641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1371/2733] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:23:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:23:08.780819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=38 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:23:14.120218 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=72 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:07.301386 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d3a3-9aaf-401e-84f2-0293bfb254ae None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:07.302123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d3a3-9aaf-401e-84f2-0293bfb254ae None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:07.302123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d3a3-9aaf-401e-84f2-0293bfb254ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:07.321525 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-26e1d3a3-9aaf-401e-84f2-0293bfb254ae None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:24:07.322851 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-26e1d3a3-9aaf-401e-84f2-0293bfb254ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:07.387441 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1363/2734] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:24:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:24:07.438198 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0bf5a-f73f-4dd5-8a7d-cd3789a76b78 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:07.439735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a9e0bf5a-f73f-4dd5-8a7d-cd3789a76b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:07.516865 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0bf5a-f73f-4dd5-8a7d-cd3789a76b78 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:07.524044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b01eb2-d9c0-4b53-96f7-82e036443a1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:07.524707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b01eb2-d9c0-4b53-96f7-82e036443a1a service swift] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:07.524992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b01eb2-d9c0-4b53-96f7-82e036443a1a service swift] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:07.525264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b01eb2-d9c0-4b53-96f7-82e036443a1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:07.534956 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b01eb2-d9c0-4b53-96f7-82e036443a1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:07.537209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b01eb2-d9c0-4b53-96f7-82e036443a1a service swift] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:07.545818 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1372/2735] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:24:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:24:17.380241 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=25 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:17.542623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=32 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:21.531240 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353a6a4-d41e-42ce-90f8-f250115c33d0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:21.531768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353a6a4-d41e-42ce-90f8-f250115c33d0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:21.531768 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353a6a4-d41e-42ce-90f8-f250115c33d0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:21.533256 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1364/2736] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 4 09:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 04 09:24:21.538461 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423c930-7964-46c1-80d0-ad8932de2950 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:21.538923 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423c930-7964-46c1-80d0-ad8932de2950 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:21.539047 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423c930-7964-46c1-80d0-ad8932de2950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:21.558383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-0423c930-7964-46c1-80d0-ad8932de2950 None None] MFA Rules not processed for user `739f3b5eaef542c29a0cf0b475c0d965`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:24:21.559595 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0423c930-7964-46c1-80d0-ad8932de2950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:21.629637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1373/2737] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 4 09:24:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:24:21.641253 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353a6a4-d41e-42ce-90f8-f250115c33d0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:21.643144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f353a6a4-d41e-42ce-90f8-f250115c33d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:21.707529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353a6a4-d41e-42ce-90f8-f250115c33d0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:21.711497 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148448c-5075-4581-90b2-9c2d68a9361c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a90e384504e8d80264d6bf20f1d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:21.712121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148448c-5075-4581-90b2-9c2d68a9361c service glance] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:21.712399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148448c-5075-4581-90b2-9c2d68a9361c service glance] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:21.712672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148448c-5075-4581-90b2-9c2d68a9361c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:21.723680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148448c-5075-4581-90b2-9c2d68a9361c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:21.726217 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148448c-5075-4581-90b2-9c2d68a9361c service glance] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:21.742977 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1365/2738] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:24:23.543587 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85f0ee-fb47-40a1-9ddc-6dbe7510069f None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:23.544085 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85f0ee-fb47-40a1-9ddc-6dbe7510069f None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:23.544085 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85f0ee-fb47-40a1-9ddc-6dbe7510069f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:23.562835 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9d85f0ee-fb47-40a1-9ddc-6dbe7510069f None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:24:23.564096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9d85f0ee-fb47-40a1-9ddc-6dbe7510069f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:23.572734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1374/2739] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 4 09:24:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:24:23.585407 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148448c-5075-4581-90b2-9c2d68a9361c service glance] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:23.601609 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148448c-5075-4581-90b2-9c2d68a9361c service glance] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:23.607471 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18698d6f-38ce-4c43-b545-a285186dd9b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:23.608108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18698d6f-38ce-4c43-b545-a285186dd9b8 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:23.608394 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18698d6f-38ce-4c43-b545-a285186dd9b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:23.608670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18698d6f-38ce-4c43-b545-a285186dd9b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:23.617780 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18698d6f-38ce-4c43-b545-a285186dd9b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:23.619380 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18698d6f-38ce-4c43-b545-a285186dd9b8 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:23.627750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1366/2740] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:24:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:24:33.557336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=34 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:33.624434 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=32 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:38.842645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ee9c-7e71-4ebb-b977-eb3427cc2349 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:38.843180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ee9c-7e71-4ebb-b977-eb3427cc2349 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:38.843180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552ee9c-7e71-4ebb-b977-eb3427cc2349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:38.861777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2552ee9c-7e71-4ebb-b977-eb3427cc2349 None None] MFA Rules not processed for user `6bb6d63553794d10a8cba042f8560348`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:24:38.862988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2552ee9c-7e71-4ebb-b977-eb3427cc2349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:38.871111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1375/2741] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 4 09:24:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:24:38.886114 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18698d6f-38ce-4c43-b545-a285186dd9b8 service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:38.898814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18698d6f-38ce-4c43-b545-a285186dd9b8 service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:38.904793 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add42d5d-4452-4f12-98ee-064a602e1da9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbb65dbfadf4c63a272f5318efa875a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:38.905440 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add42d5d-4452-4f12-98ee-064a602e1da9 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:38.905729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add42d5d-4452-4f12-98ee-064a602e1da9 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:38.906042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add42d5d-4452-4f12-98ee-064a602e1da9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:38.915077 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add42d5d-4452-4f12-98ee-064a602e1da9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:38.916669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add42d5d-4452-4f12-98ee-064a602e1da9 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:38.924991 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1367/2742] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 4 09:24:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 04 09:24:48.856831 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:48.921845 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:24:50.542444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552ee9c-7e71-4ebb-b977-eb3427cc2349 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:50.543887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2552ee9c-7e71-4ebb-b977-eb3427cc2349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:50.604171 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552ee9c-7e71-4ebb-b977-eb3427cc2349 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:50.608083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a41a2c-27fd-4c56-a26f-83b351d1e3fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:50.608846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41a2c-27fd-4c56-a26f-83b351d1e3fc service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:50.609219 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41a2c-27fd-4c56-a26f-83b351d1e3fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:50.609562 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a41a2c-27fd-4c56-a26f-83b351d1e3fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:50.611712 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a2a41a2c-27fd-4c56-a26f-83b351d1e3fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:50.626300 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a41a2c-27fd-4c56-a26f-83b351d1e3fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:50.627736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a41a2c-27fd-4c56-a26f-83b351d1e3fc service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:50.639594 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1376/2743] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:24:51.819721 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add42d5d-4452-4f12-98ee-064a602e1da9 service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:24:51.830264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add42d5d-4452-4f12-98ee-064a602e1da9 service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:24:51.834637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3b368-373a-4610-87a4-f3a973f4480d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:24:51.835293 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3b368-373a-4610-87a4-f3a973f4480d service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:24:51.835605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3b368-373a-4610-87a4-f3a973f4480d service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:24:51.835981 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3b368-373a-4610-87a4-f3a973f4480d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:24:51.837953 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c0e3b368-373a-4610-87a4-f3a973f4480d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:24:51.853092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3b368-373a-4610-87a4-f3a973f4480d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:24:51.854571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3b368-373a-4610-87a4-f3a973f4480d service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:24:51.865779 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1368/2744] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:24:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:00.634788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=26 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:01.860844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:02.815049 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a41a2c-27fd-4c56-a26f-83b351d1e3fc service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:02.824193 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a41a2c-27fd-4c56-a26f-83b351d1e3fc service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:02.828054 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e75634-4deb-49be-9d59-9e9a9c0af342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:02.828700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e75634-4deb-49be-9d59-9e9a9c0af342 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:02.828982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e75634-4deb-49be-9d59-9e9a9c0af342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:02.829260 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e75634-4deb-49be-9d59-9e9a9c0af342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:02.831083 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d4e75634-4deb-49be-9d59-9e9a9c0af342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:02.896590 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e75634-4deb-49be-9d59-9e9a9c0af342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:02.898112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e75634-4deb-49be-9d59-9e9a9c0af342 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:02.908662 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1377/2745] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:09.314945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3b368-373a-4610-87a4-f3a973f4480d service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:09.327332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3b368-373a-4610-87a4-f3a973f4480d service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:09.333469 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f382d93-47c5-459f-bc16-b6013e27b404 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:09.334305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f382d93-47c5-459f-bc16-b6013e27b404 service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:09.334666 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f382d93-47c5-459f-bc16-b6013e27b404 service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:09.335011 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f382d93-47c5-459f-bc16-b6013e27b404 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:09.336965 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7f382d93-47c5-459f-bc16-b6013e27b404 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:09.350281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f382d93-47c5-459f-bc16-b6013e27b404 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:09.351903 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f382d93-47c5-459f-bc16-b6013e27b404 service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:09.363222 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1369/2746] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:09.380020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e75634-4deb-49be-9d59-9e9a9c0af342 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:09.381731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d4e75634-4deb-49be-9d59-9e9a9c0af342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:09.394836 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e75634-4deb-49be-9d59-9e9a9c0af342 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:09.401199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:09.402321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:09.402713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:09.403108 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] PATH_INFO: `/v3/users/f884304cbb764a8ab4927c3ba8db892d` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:09.404857 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f884304cbb764a8ab4927c3ba8db892d)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:09.406889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:09.462167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f884304cbb764a8ab4927c3ba8db892d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:09.462827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f884304cbb764a8ab4927c3ba8db892d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:09.472138 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f884304cbb764a8ab4927c3ba8db892d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:09.479285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f884304cbb764a8ab4927c3ba8db892d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:09.479930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f884304cbb764a8ab4927c3ba8db892d'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:09.497357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1378/2747] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:25:09 2022] DELETE /identity/v3/users/f884304cbb764a8ab4927c3ba8db892d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:09.698376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f382d93-47c5-459f-bc16-b6013e27b404 service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:09.711956 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f382d93-47c5-459f-bc16-b6013e27b404 service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:09.774071 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:09.774996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:09.775282 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:09.775555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] PATH_INFO: `/v3/projects/dc462a5ff5f649cdb9a24f2232855724` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:09.776812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc462a5ff5f649cdb9a24f2232855724)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:09.778843 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:09.828437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc462a5ff5f649cdb9a24f2232855724', 'request_id': 'req-77adf2e3-96cf-48f9-8a14-19041db2c1a1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:09.838186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc462a5ff5f649cdb9a24f2232855724', 'request_id': 'req-77adf2e3-96cf-48f9-8a14-19041db2c1a1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:09.846831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc462a5ff5f649cdb9a24f2232855724', 'request_id': 'req-77adf2e3-96cf-48f9-8a14-19041db2c1a1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:09.859509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] The token cache is being invalidate because project dc462a5ff5f649cdb9a24f2232855724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:25:09.859799 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:09.860522 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:09.862258 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1370/2748] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:25:09 2022] DELETE /identity/v3/projects/dc462a5ff5f649cdb9a24f2232855724 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:25:09.866766 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:09.868356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:09.882609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606330b-2620-4f8c-b8e7-8668eeef9b4d admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:09.947072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:09.947859 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:09.948136 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:09.948404 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] PATH_INFO: `/v3/users/e814866b17714077bd0f6f501a49dbfb` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:09.949601 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e814866b17714077bd0f6f501a49dbfb)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:09.951564 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:09.957269 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:10.001074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e814866b17714077bd0f6f501a49dbfb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:10.001743 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e814866b17714077bd0f6f501a49dbfb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:10.010290 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e814866b17714077bd0f6f501a49dbfb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:10.017531 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e814866b17714077bd0f6f501a49dbfb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:10.018197 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e814866b17714077bd0f6f501a49dbfb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:10.036732 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1379/2749] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:25:09 2022] DELETE /identity/v3/users/e814866b17714077bd0f6f501a49dbfb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:10.227443 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:10.228983 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:10.254291 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adf2e3-96cf-48f9-8a14-19041db2c1a1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:10.312622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:10.313494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:10.313778 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:10.314085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] PATH_INFO: `/v3/projects/3a47285027a241a6bf6e9aee8b5bde37` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:10.315320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a47285027a241a6bf6e9aee8b5bde37)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:10.317286 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:10.339921 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:10.372748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a47285027a241a6bf6e9aee8b5bde37', 'request_id': 'req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:10.382871 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a47285027a241a6bf6e9aee8b5bde37', 'request_id': 'req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:10.389570 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a47285027a241a6bf6e9aee8b5bde37', 'request_id': 'req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:10.406747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] The token cache is being invalidate because project 3a47285027a241a6bf6e9aee8b5bde37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:25:10.407020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:10.407822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:10.409614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1371/2750] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:25:10 2022] DELETE /identity/v3/projects/3a47285027a241a6bf6e9aee8b5bde37 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:17.179795 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:17.181342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:17.255929 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c9bfe-ab89-4fd8-91b0-36bf2667a88c admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:17.333416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d83502-3102-4454-ab66-76ee70c16785 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:17.334117 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d83502-3102-4454-ab66-76ee70c16785 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:17.334398 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d83502-3102-4454-ab66-76ee70c16785 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:17.334668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d83502-3102-4454-ab66-76ee70c16785 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:17.336694 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-83d83502-3102-4454-ab66-76ee70c16785 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:17.358738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:17.406415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d83502-3102-4454-ab66-76ee70c16785 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:17.408093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d83502-3102-4454-ab66-76ee70c16785 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:17.419267 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1380/2751] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:20.404788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:20.512534 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:20.513923 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:20.573108 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cdef2-9874-4fe0-8c44-471e9cb19a06 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:20.577464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0222c-0a62-4ef1-9d4c-09c6483b97eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:20.578126 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0222c-0a62-4ef1-9d4c-09c6483b97eb service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:20.578409 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0222c-0a62-4ef1-9d4c-09c6483b97eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:20.578683 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0222c-0a62-4ef1-9d4c-09c6483b97eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:20.580812 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0af0222c-0a62-4ef1-9d4c-09c6483b97eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:20.654890 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0222c-0a62-4ef1-9d4c-09c6483b97eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:20.656520 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0222c-0a62-4ef1-9d4c-09c6483b97eb service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:20.667117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1372/2752] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:23.845090 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d83502-3102-4454-ab66-76ee70c16785 service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:23.846702 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-83d83502-3102-4454-ab66-76ee70c16785 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:23.863142 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d83502-3102-4454-ab66-76ee70c16785 service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:23.869599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd35d4-d6c4-4ef5-8670-50e7fe36be6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:23.870507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd35d4-d6c4-4ef5-8670-50e7fe36be6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:23.870802 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd35d4-d6c4-4ef5-8670-50e7fe36be6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:23.871079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd35d4-d6c4-4ef5-8670-50e7fe36be6c admin admin] PATH_INFO: `/v3/users/bcad5b2163e647f386b482731e1de798` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:23.872424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd35d4-d6c4-4ef5-8670-50e7fe36be6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcad5b2163e647f386b482731e1de798)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:23.874417 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd35d4-d6c4-4ef5-8670-50e7fe36be6c admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:23.935980 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-81fd35d4-d6c4-4ef5-8670-50e7fe36be6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcad5b2163e647f386b482731e1de798'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:23.936650 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-81fd35d4-d6c4-4ef5-8670-50e7fe36be6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcad5b2163e647f386b482731e1de798'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:23.945291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-81fd35d4-d6c4-4ef5-8670-50e7fe36be6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcad5b2163e647f386b482731e1de798'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:23.952682 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-81fd35d4-d6c4-4ef5-8670-50e7fe36be6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcad5b2163e647f386b482731e1de798'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:23.953342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-81fd35d4-d6c4-4ef5-8670-50e7fe36be6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcad5b2163e647f386b482731e1de798'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:23.971391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1381/2753] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:25:23 2022] DELETE /identity/v3/users/bcad5b2163e647f386b482731e1de798 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:24.153861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0222c-0a62-4ef1-9d4c-09c6483b97eb service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:24.164399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0222c-0a62-4ef1-9d4c-09c6483b97eb service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:24.223735 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:24.224602 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:24.224892 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:24.225167 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] PATH_INFO: `/v3/projects/4677b4c393ff41a09c922467365e06f0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:24.226422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4677b4c393ff41a09c922467365e06f0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:24.228569 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:24.278960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4677b4c393ff41a09c922467365e06f0', 'request_id': 'req-730c5441-34df-48e4-a89c-8c1b7a7d818d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:24.281369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:24.287788 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4677b4c393ff41a09c922467365e06f0', 'request_id': 'req-730c5441-34df-48e4-a89c-8c1b7a7d818d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:24.295571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4677b4c393ff41a09c922467365e06f0', 'request_id': 'req-730c5441-34df-48e4-a89c-8c1b7a7d818d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:24.306684 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] The token cache is being invalidate because project 4677b4c393ff41a09c922467365e06f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:25:24.306972 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:24.307603 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-730c5441-34df-48e4-a89c-8c1b7a7d818d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:25:24.309181 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1373/2754] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:25:24 2022] DELETE /identity/v3/projects/4677b4c393ff41a09c922467365e06f0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:24.314209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804055a-d66c-4c6d-b0ab-639b5cefb878 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:24.314507 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804055a-d66c-4c6d-b0ab-639b5cefb878 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:24.314786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804055a-d66c-4c6d-b0ab-639b5cefb878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:24.333902 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9804055a-d66c-4c6d-b0ab-639b5cefb878 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:24.335140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9804055a-d66c-4c6d-b0ab-639b5cefb878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:24.397498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1382/2755] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:25:24.406130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d049376-95f4-419f-841e-59039c5aafdf None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:24.406428 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d049376-95f4-419f-841e-59039c5aafdf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:24.406711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d049376-95f4-419f-841e-59039c5aafdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:24.427130 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0d049376-95f4-419f-841e-59039c5aafdf None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:24.428366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0d049376-95f4-419f-841e-59039c5aafdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:24.438096 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1374/2756] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:24.442479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804055a-d66c-4c6d-b0ab-639b5cefb878 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:24.452744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804055a-d66c-4c6d-b0ab-639b5cefb878 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:24.458669 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bd205-c654-4d67-b7c2-734cf9ba09ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:24.459328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bd205-c654-4d67-b7c2-734cf9ba09ee admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:24.459613 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bd205-c654-4d67-b7c2-734cf9ba09ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:24.459889 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bd205-c654-4d67-b7c2-734cf9ba09ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:24.460620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bd205-c654-4d67-b7c2-734cf9ba09ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:25:24.460925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bd205-c654-4d67-b7c2-734cf9ba09ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:24.462615 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bd205-c654-4d67-b7c2-734cf9ba09ee admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:24.474556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1383/2757] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:25:24.479023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d049376-95f4-419f-841e-59039c5aafdf None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:24.488631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d049376-95f4-419f-841e-59039c5aafdf None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:24.494173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5773a-c653-4d21-a73c-88d4da195f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:24.494829 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5773a-c653-4d21-a73c-88d4da195f20 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:24.495115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5773a-c653-4d21-a73c-88d4da195f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:24.495395 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5773a-c653-4d21-a73c-88d4da195f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:24.496131 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5773a-c653-4d21-a73c-88d4da195f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:24.498079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5773a-c653-4d21-a73c-88d4da195f20 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:24.525675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1375/2758] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:25:24 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:25:24.529945 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bd205-c654-4d67-b7c2-734cf9ba09ee admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:24.539966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bd205-c654-4d67-b7c2-734cf9ba09ee admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:24.558301 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:24.606425 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ac333-bd80-44e9-948c-c34b61d3b790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:24.607449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ac333-bd80-44e9-948c-c34b61d3b790 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:24.607894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ac333-bd80-44e9-948c-c34b61d3b790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:24.608304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ac333-bd80-44e9-948c-c34b61d3b790 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:24.609426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ac333-bd80-44e9-948c-c34b61d3b790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:24.612478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ac333-bd80-44e9-948c-c34b61d3b790 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:24.637933 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1384/2759] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:25:24 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:25:24.642437 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5773a-c653-4d21-a73c-88d4da195f20 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:24.653833 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5773a-c653-4d21-a73c-88d4da195f20 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:24.658916 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bc437-a629-4286-a924-151c314f8769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:24.659581 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bc437-a629-4286-a924-151c314f8769 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:24.659870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bc437-a629-4286-a924-151c314f8769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:24.660133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bc437-a629-4286-a924-151c314f8769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:24.660870 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bc437-a629-4286-a924-151c314f8769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:24.662504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bc437-a629-4286-a924-151c314f8769 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:24.673530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1376/2760] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:25:24.677637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ac333-bd80-44e9-948c-c34b61d3b790 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:24.688423 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ac333-bd80-44e9-948c-c34b61d3b790 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:24.694466 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c838b-9b84-489d-a21b-6ed99eb78d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:24.695160 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c838b-9b84-489d-a21b-6ed99eb78d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:24.695443 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c838b-9b84-489d-a21b-6ed99eb78d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:24.695719 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c838b-9b84-489d-a21b-6ed99eb78d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:24.696459 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c838b-9b84-489d-a21b-6ed99eb78d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:24.698351 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c838b-9b84-489d-a21b-6ed99eb78d7e admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:24.709449 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1385/2761] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:25:24.714430 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bc437-a629-4286-a924-151c314f8769 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:24.726178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bc437-a629-4286-a924-151c314f8769 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:24.732145 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de2005-6476-4317-a76d-5271d492f523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:24.733023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de2005-6476-4317-a76d-5271d492f523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:24.733339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de2005-6476-4317-a76d-5271d492f523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:24.733617 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de2005-6476-4317-a76d-5271d492f523 admin admin] PATH_INFO: `/v3/projects/9157e722ff614bc2916f1971cde517ef/users/6852ed4b2e53480b9e5719c7575c8eb9/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:24.750115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de2005-6476-4317-a76d-5271d492f523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9157e722ff614bc2916f1971cde517ef, user_id=6852ed4b2e53480b9e5719c7575c8eb9, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:24.752769 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de2005-6476-4317-a76d-5271d492f523 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:24.767396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1377/2762] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:25:24 2022] PUT /identity/v3/projects/9157e722ff614bc2916f1971cde517ef/users/6852ed4b2e53480b9e5719c7575c8eb9/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:25:24.772526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff3e9-b9ac-4e48-a417-845ccb2df217 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:24.773203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff3e9-b9ac-4e48-a417-845ccb2df217 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:24.773551 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff3e9-b9ac-4e48-a417-845ccb2df217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:24.792315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-1a8ff3e9-b9ac-4e48-a417-845ccb2df217 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:24.793457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-1a8ff3e9-b9ac-4e48-a417-845ccb2df217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:24.857569 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1386/2763] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:24.867536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de2005-6476-4317-a76d-5271d492f523 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:24.870068 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-14de2005-6476-4317-a76d-5271d492f523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:24.932616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de2005-6476-4317-a76d-5271d492f523 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:24.937149 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6bb7a-d87b-48af-98e5-8ad87506ff04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:24.937802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6bb7a-d87b-48af-98e5-8ad87506ff04 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:24.938121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6bb7a-d87b-48af-98e5-8ad87506ff04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:24.938405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6bb7a-d87b-48af-98e5-8ad87506ff04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:24.948618 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6bb7a-d87b-48af-98e5-8ad87506ff04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:24.950098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6bb7a-d87b-48af-98e5-8ad87506ff04 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:24.960005 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1378/2764] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:28.897776 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999cc14-3de8-4546-bcd2-c6ed9708f0b2 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:28.898336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999cc14-3de8-4546-bcd2-c6ed9708f0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:28.898336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999cc14-3de8-4546-bcd2-c6ed9708f0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:28.929286 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-d999cc14-3de8-4546-bcd2-c6ed9708f0b2 None None] MFA Rules not processed for user `6852ed4b2e53480b9e5719c7575c8eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:28.930938 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d999cc14-3de8-4546-bcd2-c6ed9708f0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:29.006546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1387/2765] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:25:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:29.021922 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6bb7a-d87b-48af-98e5-8ad87506ff04 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:29.023826 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-10d6bb7a-d87b-48af-98e5-8ad87506ff04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:29.104703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6bb7a-d87b-48af-98e5-8ad87506ff04 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:29.110464 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9695ab1-0b32-4841-af16-c20a7ee743ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:29.111129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9695ab1-0b32-4841-af16-c20a7ee743ab service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:29.111414 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9695ab1-0b32-4841-af16-c20a7ee743ab service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:29.111690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9695ab1-0b32-4841-af16-c20a7ee743ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:29.121495 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9695ab1-0b32-4841-af16-c20a7ee743ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:29.123164 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9695ab1-0b32-4841-af16-c20a7ee743ab service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:29.136504 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1379/2766] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:30.437642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869f7be-ea3c-4f43-a6c8-2f01da41a935 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:30.438238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869f7be-ea3c-4f43-a6c8-2f01da41a935 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:30.438238 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869f7be-ea3c-4f43-a6c8-2f01da41a935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:30.456994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-4869f7be-ea3c-4f43-a6c8-2f01da41a935 None None] MFA Rules not processed for user `6852ed4b2e53480b9e5719c7575c8eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:25:30.458354 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4869f7be-ea3c-4f43-a6c8-2f01da41a935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:30.465619 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1388/2767] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:25:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:25:30.781741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9695ab1-0b32-4841-af16-c20a7ee743ab service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:30.793636 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:30.802729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9695ab1-0b32-4841-af16-c20a7ee743ab service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:30.807028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77372c-8db3-4274-aa6e-7cc47d3a7db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:30.807690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77372c-8db3-4274-aa6e-7cc47d3a7db5 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:30.807976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77372c-8db3-4274-aa6e-7cc47d3a7db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:30.808271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77372c-8db3-4274-aa6e-7cc47d3a7db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:30.810313 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7e77372c-8db3-4274-aa6e-7cc47d3a7db5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:30.820983 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77372c-8db3-4274-aa6e-7cc47d3a7db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:30.822599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77372c-8db3-4274-aa6e-7cc47d3a7db5 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:30.832174 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1380/2768] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:31.308504 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869f7be-ea3c-4f43-a6c8-2f01da41a935 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:31.310846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-4869f7be-ea3c-4f43-a6c8-2f01da41a935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:31.348546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:31.420214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869f7be-ea3c-4f43-a6c8-2f01da41a935 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:31.431833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407c0d8-10eb-4901-940e-dcbce939684a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:31.432523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407c0d8-10eb-4901-940e-dcbce939684a service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:31.432811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407c0d8-10eb-4901-940e-dcbce939684a service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:31.433089 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407c0d8-10eb-4901-940e-dcbce939684a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:31.434988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f407c0d8-10eb-4901-940e-dcbce939684a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:31.447384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407c0d8-10eb-4901-940e-dcbce939684a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:31.449124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407c0d8-10eb-4901-940e-dcbce939684a service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:31.460681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1389/2769] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:33.620420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77372c-8db3-4274-aa6e-7cc47d3a7db5 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:33.632237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77372c-8db3-4274-aa6e-7cc47d3a7db5 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:33.636535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f5b3b-2a97-4877-b5bb-6fe34317495b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:33.638763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f5b3b-2a97-4877-b5bb-6fe34317495b service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:33.639074 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f5b3b-2a97-4877-b5bb-6fe34317495b service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:33.639349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f5b3b-2a97-4877-b5bb-6fe34317495b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:33.641447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-512f5b3b-2a97-4877-b5bb-6fe34317495b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:33.653515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f5b3b-2a97-4877-b5bb-6fe34317495b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:33.655042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f5b3b-2a97-4877-b5bb-6fe34317495b service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:33.667657 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1381/2770] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:34.325383 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407c0d8-10eb-4901-940e-dcbce939684a service placement] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:25:34.327714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f407c0d8-10eb-4901-940e-dcbce939684a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:34.347062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407c0d8-10eb-4901-940e-dcbce939684a service placement] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:25:34.353926 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2d831-4e05-4519-bfe9-fec70734d315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:25:34.354570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2d831-4e05-4519-bfe9-fec70734d315 service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:25:34.354853 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2d831-4e05-4519-bfe9-fec70734d315 service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:25:34.355188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2d831-4e05-4519-bfe9-fec70734d315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:25:34.357264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-90d2d831-4e05-4519-bfe9-fec70734d315 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:25:34.372180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2d831-4e05-4519-bfe9-fec70734d315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:25:34.374366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2d831-4e05-4519-bfe9-fec70734d315 service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:25:34.386811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1390/2771] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:25:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:25:43.663462 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=10 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:25:44.382340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=28 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:28.770503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931c736-e6d6-4a85-9f75-21b58ee1ab0e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:28.771090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931c736-e6d6-4a85-9f75-21b58ee1ab0e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:28.771090 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931c736-e6d6-4a85-9f75-21b58ee1ab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:28.797137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e931c736-e6d6-4a85-9f75-21b58ee1ab0e None None] MFA Rules not processed for user `6852ed4b2e53480b9e5719c7575c8eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:26:28.798888 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e931c736-e6d6-4a85-9f75-21b58ee1ab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:28.808607 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1382/2772] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:26:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:26:37.605074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2d831-4e05-4519-bfe9-fec70734d315 service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:26:37.619610 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2d831-4e05-4519-bfe9-fec70734d315 service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:26:37.626332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133c67d-adb9-4eb4-b516-ba4ce0c7f6de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:26:37.627076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133c67d-adb9-4eb4-b516-ba4ce0c7f6de service nova] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:26:37.627367 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133c67d-adb9-4eb4-b516-ba4ce0c7f6de service nova] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:26:37.627677 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133c67d-adb9-4eb4-b516-ba4ce0c7f6de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:26:37.629668 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-0133c67d-adb9-4eb4-b516-ba4ce0c7f6de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:26:37.644266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133c67d-adb9-4eb4-b516-ba4ce0c7f6de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:26:37.646365 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133c67d-adb9-4eb4-b516-ba4ce0c7f6de service nova] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:26:37.659607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1391/2773] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:26:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:26:38.789226 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:26:47.654368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=6 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:30.073672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db1433-6774-45c1-b80c-1527447446d4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:30.074323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db1433-6774-45c1-b80c-1527447446d4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:30.074436 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db1433-6774-45c1-b80c-1527447446d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:30.103298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d3db1433-6774-45c1-b80c-1527447446d4 None None] MFA Rules not processed for user `6852ed4b2e53480b9e5719c7575c8eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:27:30.104543 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d3db1433-6774-45c1-b80c-1527447446d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:30.128822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1383/2774] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:27:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:27:30.460082 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133c67d-adb9-4eb4-b516-ba4ce0c7f6de service nova] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:27:30.499903 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133c67d-adb9-4eb4-b516-ba4ce0c7f6de service nova] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:27:30.503686 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8037e78-8cc3-4dcd-a115-b38e08c243d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:27:30.504336 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8037e78-8cc3-4dcd-a115-b38e08c243d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:27:30.504609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8037e78-8cc3-4dcd-a115-b38e08c243d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:27:30.504880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8037e78-8cc3-4dcd-a115-b38e08c243d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:27:30.506867 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d8037e78-8cc3-4dcd-a115-b38e08c243d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:27:30.532348 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8037e78-8cc3-4dcd-a115-b38e08c243d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:27:30.534817 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8037e78-8cc3-4dcd-a115-b38e08c243d0 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:27:30.545572 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1392/2775] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:27:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:27:40.126729 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=15 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:27:40.540851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=23 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:28:00.098612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db1433-6774-45c1-b80c-1527447446d4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:00.129378 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db1433-6774-45c1-b80c-1527447446d4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:00.135509 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72928bee-7e12-422c-af6f-f01cefd43ba9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:00.136228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72928bee-7e12-422c-af6f-f01cefd43ba9 service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:00.136518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72928bee-7e12-422c-af6f-f01cefd43ba9 service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:00.136796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72928bee-7e12-422c-af6f-f01cefd43ba9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:00.139175 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-72928bee-7e12-422c-af6f-f01cefd43ba9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:00.165837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72928bee-7e12-422c-af6f-f01cefd43ba9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:00.167478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72928bee-7e12-422c-af6f-f01cefd43ba9 service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:00.178228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1384/2776] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:28:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:28:10.173847 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=18 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:28:34.657365 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db3b8e-32a5-4893-aee7-ed982f500ea3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:34.658086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db3b8e-32a5-4893-aee7-ed982f500ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:34.658086 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db3b8e-32a5-4893-aee7-ed982f500ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:34.677112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-a4db3b8e-32a5-4893-aee7-ed982f500ea3 None None] MFA Rules not processed for user `6852ed4b2e53480b9e5719c7575c8eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:28:34.678683 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a4db3b8e-32a5-4893-aee7-ed982f500ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:34.686960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1393/2777] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:28:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:28:44.671571 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:28:46.222672 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72928bee-7e12-422c-af6f-f01cefd43ba9 service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:28:46.253041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72928bee-7e12-422c-af6f-f01cefd43ba9 service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:28:46.259104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728da149-b40b-4990-92aa-5d63a66b328e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '739f3b5eaef542c29a0cf0b475c0d965', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:28:46.259763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728da149-b40b-4990-92aa-5d63a66b328e service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:28:46.260051 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728da149-b40b-4990-92aa-5d63a66b328e service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:28:46.260326 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728da149-b40b-4990-92aa-5d63a66b328e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:28:46.262264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-728da149-b40b-4990-92aa-5d63a66b328e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:28:46.288662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728da149-b40b-4990-92aa-5d63a66b328e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:28:46.290399 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728da149-b40b-4990-92aa-5d63a66b328e service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:28:46.301160 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1385/2778] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:28:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:28:56.296614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=18 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:29:46.325827 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db3b8e-32a5-4893-aee7-ed982f500ea3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:46.357560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db3b8e-32a5-4893-aee7-ed982f500ea3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:46.363734 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21560997-89fb-4018-ae74-49eeb8be442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:46.364752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21560997-89fb-4018-ae74-49eeb8be442d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:46.365084 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21560997-89fb-4018-ae74-49eeb8be442d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:46.365400 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21560997-89fb-4018-ae74-49eeb8be442d admin admin] PATH_INFO: `/v3/users/6852ed4b2e53480b9e5719c7575c8eb9` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:46.366769 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21560997-89fb-4018-ae74-49eeb8be442d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6852ed4b2e53480b9e5719c7575c8eb9)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:46.369490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21560997-89fb-4018-ae74-49eeb8be442d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:46.426781 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-21560997-89fb-4018-ae74-49eeb8be442d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6852ed4b2e53480b9e5719c7575c8eb9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:29:46.427608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-21560997-89fb-4018-ae74-49eeb8be442d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6852ed4b2e53480b9e5719c7575c8eb9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:29:46.436231 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-21560997-89fb-4018-ae74-49eeb8be442d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6852ed4b2e53480b9e5719c7575c8eb9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:29:46.443283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-21560997-89fb-4018-ae74-49eeb8be442d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6852ed4b2e53480b9e5719c7575c8eb9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:29:46.443931 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-21560997-89fb-4018-ae74-49eeb8be442d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6852ed4b2e53480b9e5719c7575c8eb9'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:29:46.462025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1394/2779] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:29:46 2022] DELETE /identity/v3/users/6852ed4b2e53480b9e5719c7575c8eb9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:29:46.657564 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728da149-b40b-4990-92aa-5d63a66b328e service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:46.669270 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728da149-b40b-4990-92aa-5d63a66b328e service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:46.729129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:46.730346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:46.730750 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:46.731102 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] PATH_INFO: `/v3/projects/9157e722ff614bc2916f1971cde517ef` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:46.733106 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9157e722ff614bc2916f1971cde517ef)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:46.735942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:46.788945 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9157e722ff614bc2916f1971cde517ef', 'request_id': 'req-0e729de2-d77a-4494-a650-f57a015cd302'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:29:46.798236 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9157e722ff614bc2916f1971cde517ef', 'request_id': 'req-0e729de2-d77a-4494-a650-f57a015cd302'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:29:46.804880 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9157e722ff614bc2916f1971cde517ef', 'request_id': 'req-0e729de2-d77a-4494-a650-f57a015cd302'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:29:46.828910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] The token cache is being invalidate because project 9157e722ff614bc2916f1971cde517ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:29:46.829207 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:29:46.829981 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e729de2-d77a-4494-a650-f57a015cd302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:29:46.831663 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1386/2780] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:29:46 2022] DELETE /identity/v3/projects/9157e722ff614bc2916f1971cde517ef => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:29:46.837213 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009097e3-8702-4bbf-a214-077c2fe29a73 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:46.837512 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009097e3-8702-4bbf-a214-077c2fe29a73 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:46.837825 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009097e3-8702-4bbf-a214-077c2fe29a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:46.897987 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-009097e3-8702-4bbf-a214-077c2fe29a73 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:29:46.899241 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-009097e3-8702-4bbf-a214-077c2fe29a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:29:46.964811 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1395/2781] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:29:46.973706 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21599fb1-1c54-49d5-b7d1-ca48a5a8fe4c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:46.974112 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21599fb1-1c54-49d5-b7d1-ca48a5a8fe4c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:46.974508 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21599fb1-1c54-49d5-b7d1-ca48a5a8fe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:46.994066 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-21599fb1-1c54-49d5-b7d1-ca48a5a8fe4c None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:29:46.995306 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-21599fb1-1c54-49d5-b7d1-ca48a5a8fe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:29:47.005462 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1387/2782] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:29:47.009966 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009097e3-8702-4bbf-a214-077c2fe29a73 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.020537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009097e3-8702-4bbf-a214-077c2fe29a73 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:47.025610 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e247d-bd2e-4a7c-af7b-7ec3e4bd4f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:47.026323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e247d-bd2e-4a7c-af7b-7ec3e4bd4f93 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.026612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e247d-bd2e-4a7c-af7b-7ec3e4bd4f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.026894 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e247d-bd2e-4a7c-af7b-7ec3e4bd4f93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.027631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e247d-bd2e-4a7c-af7b-7ec3e4bd4f93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:29:47.027948 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e247d-bd2e-4a7c-af7b-7ec3e4bd4f93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:47.029586 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e247d-bd2e-4a7c-af7b-7ec3e4bd4f93 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:47.041605 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1396/2783] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:29:47.046261 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21599fb1-1c54-49d5-b7d1-ca48a5a8fe4c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.055966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21599fb1-1c54-49d5-b7d1-ca48a5a8fe4c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:47.061816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea780c-dc63-4d53-a6a3-fe626f5f8fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:47.062555 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea780c-dc63-4d53-a6a3-fe626f5f8fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.062840 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea780c-dc63-4d53-a6a3-fe626f5f8fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.063119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea780c-dc63-4d53-a6a3-fe626f5f8fe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.063822 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea780c-dc63-4d53-a6a3-fe626f5f8fe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:47.065760 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea780c-dc63-4d53-a6a3-fe626f5f8fe9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:47.094835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1388/2784] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:29:47 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:29:47.099235 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e247d-bd2e-4a7c-af7b-7ec3e4bd4f93 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.109427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e247d-bd2e-4a7c-af7b-7ec3e4bd4f93 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:47.118632 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:29:47.172717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3a113-5e58-49a7-ac43-a09e0877e9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:47.173401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3a113-5e58-49a7-ac43-a09e0877e9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.173688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3a113-5e58-49a7-ac43-a09e0877e9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.174008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3a113-5e58-49a7-ac43-a09e0877e9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.174733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3a113-5e58-49a7-ac43-a09e0877e9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:47.176733 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3a113-5e58-49a7-ac43-a09e0877e9c9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:47.198942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1397/2785] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:29:47 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:29:47.203188 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea780c-dc63-4d53-a6a3-fe626f5f8fe9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.214101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea780c-dc63-4d53-a6a3-fe626f5f8fe9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:47.223377 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0275f-7111-4e36-ad78-df0e61f9125d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:47.224058 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0275f-7111-4e36-ad78-df0e61f9125d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.224320 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0275f-7111-4e36-ad78-df0e61f9125d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.224629 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0275f-7111-4e36-ad78-df0e61f9125d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.225369 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0275f-7111-4e36-ad78-df0e61f9125d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:47.227054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0275f-7111-4e36-ad78-df0e61f9125d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:47.238225 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1389/2786] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:29:47.242635 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3a113-5e58-49a7-ac43-a09e0877e9c9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.263523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3a113-5e58-49a7-ac43-a09e0877e9c9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:47.270067 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faafd11-2298-4f8a-880e-f04faba93e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:47.270805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faafd11-2298-4f8a-880e-f04faba93e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.271153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faafd11-2298-4f8a-880e-f04faba93e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.271426 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faafd11-2298-4f8a-880e-f04faba93e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.272254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faafd11-2298-4f8a-880e-f04faba93e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:47.273947 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faafd11-2298-4f8a-880e-f04faba93e9d admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:47.285242 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1398/2787] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:29:47.289995 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0275f-7111-4e36-ad78-df0e61f9125d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.300927 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0275f-7111-4e36-ad78-df0e61f9125d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:47.307006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569ff36-67a4-4cf5-bdfe-910be8f9f4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:47.307918 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569ff36-67a4-4cf5-bdfe-910be8f9f4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.308199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569ff36-67a4-4cf5-bdfe-910be8f9f4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.308450 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569ff36-67a4-4cf5-bdfe-910be8f9f4fe admin admin] PATH_INFO: `/v3/projects/8d6d07f7be8f42f4aeff74c4ab9423bb/users/a4f0c75e003240b7a75f35609ee683bc/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.325805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569ff36-67a4-4cf5-bdfe-910be8f9f4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d6d07f7be8f42f4aeff74c4ab9423bb, user_id=a4f0c75e003240b7a75f35609ee683bc, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:47.328531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569ff36-67a4-4cf5-bdfe-910be8f9f4fe admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:47.344531 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1390/2788] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:29:47 2022] PUT /identity/v3/projects/8d6d07f7be8f42f4aeff74c4ab9423bb/users/a4f0c75e003240b7a75f35609ee683bc/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:29:47.407834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3455f74-f2cd-4544-91be-31a76841847c None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.408599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3455f74-f2cd-4544-91be-31a76841847c None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.408599 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3455f74-f2cd-4544-91be-31a76841847c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.435908 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-e3455f74-f2cd-4544-91be-31a76841847c None None] MFA Rules not processed for user `a4f0c75e003240b7a75f35609ee683bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:29:47.437103 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-e3455f74-f2cd-4544-91be-31a76841847c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:29:47.509822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1399/2789] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:29:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:29:47.516876 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569ff36-67a4-4cf5-bdfe-910be8f9f4fe admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.536446 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:29:47.593951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569ff36-67a4-4cf5-bdfe-910be8f9f4fe admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:47.599388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1978a67-f14f-4134-90c7-609c25a8b137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:47.600073 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1978a67-f14f-4134-90c7-609c25a8b137 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.600349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1978a67-f14f-4134-90c7-609c25a8b137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.600614 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1978a67-f14f-4134-90c7-609c25a8b137 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.601325 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1978a67-f14f-4134-90c7-609c25a8b137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:47.603911 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1978a67-f14f-4134-90c7-609c25a8b137 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:47.638832 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1391/2790] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:29:47 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:29:47.645695 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3455f74-f2cd-4544-91be-31a76841847c None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.659018 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3455f74-f2cd-4544-91be-31a76841847c None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:47.726508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10adb35-4589-4cf5-9c9e-5d7cc06531c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:47.727196 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10adb35-4589-4cf5-9c9e-5d7cc06531c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.727477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10adb35-4589-4cf5-9c9e-5d7cc06531c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.727755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10adb35-4589-4cf5-9c9e-5d7cc06531c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.728460 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10adb35-4589-4cf5-9c9e-5d7cc06531c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:47.730429 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10adb35-4589-4cf5-9c9e-5d7cc06531c9 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:47.753005 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1400/2791] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:29:47 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:29:47.758816 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1978a67-f14f-4134-90c7-609c25a8b137 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.770034 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1978a67-f14f-4134-90c7-609c25a8b137 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:47.775688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a4c1e-e715-434f-bcb8-c4e29bac55df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:47.776356 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a4c1e-e715-434f-bcb8-c4e29bac55df admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.776638 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a4c1e-e715-434f-bcb8-c4e29bac55df admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.776910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a4c1e-e715-434f-bcb8-c4e29bac55df admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.777637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a4c1e-e715-434f-bcb8-c4e29bac55df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:47.779292 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a4c1e-e715-434f-bcb8-c4e29bac55df admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:47.790452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1392/2792] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:29:47.795478 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10adb35-4589-4cf5-9c9e-5d7cc06531c9 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.812046 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10adb35-4589-4cf5-9c9e-5d7cc06531c9 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:47.819323 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747749b-81b7-4094-8c6d-9b2cc9572735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:47.820665 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747749b-81b7-4094-8c6d-9b2cc9572735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.821199 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747749b-81b7-4094-8c6d-9b2cc9572735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.821485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747749b-81b7-4094-8c6d-9b2cc9572735 admin admin] PATH_INFO: `/v3/projects/0baea7ab89de426eb8f1e101f3abaf36/users/070fe65127c04dfba0d64a8b55187b27/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.831172 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:29:47.839185 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747749b-81b7-4094-8c6d-9b2cc9572735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0baea7ab89de426eb8f1e101f3abaf36, user_id=070fe65127c04dfba0d64a8b55187b27, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:47.842330 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747749b-81b7-4094-8c6d-9b2cc9572735 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:47.857971 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1401/2793] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:29:47 2022] PUT /identity/v3/projects/0baea7ab89de426eb8f1e101f3abaf36/users/070fe65127c04dfba0d64a8b55187b27/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:29:47.862379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a4c1e-e715-434f-bcb8-c4e29bac55df admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.876565 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a4c1e-e715-434f-bcb8-c4e29bac55df admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:47.962796 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ecfb0-c36a-4f45-adae-794b90417db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:47.963490 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ecfb0-c36a-4f45-adae-794b90417db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:47.963777 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ecfb0-c36a-4f45-adae-794b90417db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:47.964054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ecfb0-c36a-4f45-adae-794b90417db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:47.964795 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ecfb0-c36a-4f45-adae-794b90417db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:47.966515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ecfb0-c36a-4f45-adae-794b90417db7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:47.977264 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1393/2794] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:29:47.983299 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747749b-81b7-4094-8c6d-9b2cc9572735 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:47.994731 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747749b-81b7-4094-8c6d-9b2cc9572735 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:48.000045 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c27746-0f9f-4014-b527-67e1b34d5ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:48.000960 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c27746-0f9f-4014-b527-67e1b34d5ab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:48.001239 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c27746-0f9f-4014-b527-67e1b34d5ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:48.001514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c27746-0f9f-4014-b527-67e1b34d5ab4 admin admin] PATH_INFO: `/v3/projects/0baea7ab89de426eb8f1e101f3abaf36/users/070fe65127c04dfba0d64a8b55187b27/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:48.003721 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c27746-0f9f-4014-b527-67e1b34d5ab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0baea7ab89de426eb8f1e101f3abaf36, user_id=070fe65127c04dfba0d64a8b55187b27, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:48.006417 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c27746-0f9f-4014-b527-67e1b34d5ab4 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:48.017943 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1402/2795] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:29:47 2022] PUT /identity/v3/projects/0baea7ab89de426eb8f1e101f3abaf36/users/070fe65127c04dfba0d64a8b55187b27/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:29:48.030997 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b78da-e9e5-416f-b3d5-f9944a9575be None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:48.031210 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b78da-e9e5-416f-b3d5-f9944a9575be None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:48.031488 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b78da-e9e5-416f-b3d5-f9944a9575be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:48.061784 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b07b78da-e9e5-416f-b3d5-f9944a9575be None None] MFA Rules not processed for user `070fe65127c04dfba0d64a8b55187b27`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:29:48.063397 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b07b78da-e9e5-416f-b3d5-f9944a9575be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:29:48.144623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1394/2796] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:29:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:29:48.156732 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c27746-0f9f-4014-b527-67e1b34d5ab4 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:48.158876 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b4c27746-0f9f-4014-b527-67e1b34d5ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:29:48.249693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c27746-0f9f-4014-b527-67e1b34d5ab4 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:48.253649 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75967c-c1bf-47cc-9c5f-92fab837d827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:48.254332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75967c-c1bf-47cc-9c5f-92fab837d827 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:48.254617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75967c-c1bf-47cc-9c5f-92fab837d827 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:48.254890 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75967c-c1bf-47cc-9c5f-92fab837d827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:48.265869 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75967c-c1bf-47cc-9c5f-92fab837d827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:48.267346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75967c-c1bf-47cc-9c5f-92fab837d827 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:48.277814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1403/2797] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:29:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:29:48.354574 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b78da-e9e5-416f-b3d5-f9944a9575be None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:29:48.365536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b78da-e9e5-416f-b3d5-f9944a9575be None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:29:48.369714 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c37f0d-f249-4b70-ac3d-6d753a2c3d40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:29:48.370401 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37f0d-f249-4b70-ac3d-6d753a2c3d40 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:29:48.370690 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37f0d-f249-4b70-ac3d-6d753a2c3d40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:29:48.370966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c37f0d-f249-4b70-ac3d-6d753a2c3d40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:29:48.379864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:29:48.430758 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c37f0d-f249-4b70-ac3d-6d753a2c3d40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:29:48.432423 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c37f0d-f249-4b70-ac3d-6d753a2c3d40 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:29:48.441795 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1395/2798] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:29:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:29:58.272788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=45,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:29:58.437748 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=23 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:28.404366 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75967c-c1bf-47cc-9c5f-92fab837d827 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:28.468801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75967c-c1bf-47cc-9c5f-92fab837d827 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:28.474294 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:28.475180 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:28.475462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:28.475738 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] PATH_INFO: `/v3/users/a4f0c75e003240b7a75f35609ee683bc` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:28.476955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4f0c75e003240b7a75f35609ee683bc)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:28.478930 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:28.538453 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4f0c75e003240b7a75f35609ee683bc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:28.539293 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4f0c75e003240b7a75f35609ee683bc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:28.550719 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4f0c75e003240b7a75f35609ee683bc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:28.558678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4f0c75e003240b7a75f35609ee683bc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:28.559457 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4f0c75e003240b7a75f35609ee683bc'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:28.578788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1404/2799] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:30:28 2022] DELETE /identity/v3/users/a4f0c75e003240b7a75f35609ee683bc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:28.583627 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e2b11-36d7-48c7-8c20-1e9f794c998d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:28.584172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e2b11-36d7-48c7-8c20-1e9f794c998d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:28.584172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e2b11-36d7-48c7-8c20-1e9f794c998d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:28.602835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6f0e2b11-36d7-48c7-8c20-1e9f794c998d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:28.604070 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6f0e2b11-36d7-48c7-8c20-1e9f794c998d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:28.666024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1396/2800] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:28.675993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:28.677556 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:28.737833 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d67d8c-8a83-4ddc-8960-5d44bd16a582 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:28.742285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6baf90-9a41-4de0-b015-9493290d09fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:28.742949 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6baf90-9a41-4de0-b015-9493290d09fe service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:28.743230 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6baf90-9a41-4de0-b015-9493290d09fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:28.743509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6baf90-9a41-4de0-b015-9493290d09fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:28.753492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6baf90-9a41-4de0-b015-9493290d09fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:28.755014 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6baf90-9a41-4de0-b015-9493290d09fe service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:28.765340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1405/2801] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:29.040642 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e2b11-36d7-48c7-8c20-1e9f794c998d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:29.042530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6f0e2b11-36d7-48c7-8c20-1e9f794c998d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:29.064199 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e2b11-36d7-48c7-8c20-1e9f794c998d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:29.069763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:29.070688 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:29.070976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:29.071244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] PATH_INFO: `/v3/projects/8d6d07f7be8f42f4aeff74c4ab9423bb` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:29.072452 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d6d07f7be8f42f4aeff74c4ab9423bb)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:29.074480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:29.122640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d6d07f7be8f42f4aeff74c4ab9423bb', 'request_id': 'req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.133032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d6d07f7be8f42f4aeff74c4ab9423bb', 'request_id': 'req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.140681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d6d07f7be8f42f4aeff74c4ab9423bb', 'request_id': 'req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.151379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] The token cache is being invalidate because project 8d6d07f7be8f42f4aeff74c4ab9423bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:30:29.151670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.152355 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.153993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1397/2802] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:30:29 2022] DELETE /identity/v3/projects/8d6d07f7be8f42f4aeff74c4ab9423bb => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:29.159124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6baf90-9a41-4de0-b015-9493290d09fe service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:29.168955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6baf90-9a41-4de0-b015-9493290d09fe service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:29.187475 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:29.229419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1df0d-b6c1-432b-8b9b-7509fbae3003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:29.230356 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1df0d-b6c1-432b-8b9b-7509fbae3003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:29.230643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1df0d-b6c1-432b-8b9b-7509fbae3003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:29.230932 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1df0d-b6c1-432b-8b9b-7509fbae3003 admin admin] PATH_INFO: `/v3/users/070fe65127c04dfba0d64a8b55187b27` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:29.232508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1df0d-b6c1-432b-8b9b-7509fbae3003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=070fe65127c04dfba0d64a8b55187b27)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:29.234626 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1df0d-b6c1-432b-8b9b-7509fbae3003 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:29.283623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c7b1df0d-b6c1-432b-8b9b-7509fbae3003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '070fe65127c04dfba0d64a8b55187b27'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.284620 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c7b1df0d-b6c1-432b-8b9b-7509fbae3003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '070fe65127c04dfba0d64a8b55187b27'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.292285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c7b1df0d-b6c1-432b-8b9b-7509fbae3003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '070fe65127c04dfba0d64a8b55187b27'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.299600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c7b1df0d-b6c1-432b-8b9b-7509fbae3003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '070fe65127c04dfba0d64a8b55187b27'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.300253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-c7b1df0d-b6c1-432b-8b9b-7509fbae3003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '070fe65127c04dfba0d64a8b55187b27'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.319673 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1406/2803] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:30:29 2022] DELETE /identity/v3/users/070fe65127c04dfba0d64a8b55187b27 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:29.604111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:29.605741 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:29.630006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa4114-db6b-4814-9cf3-47f7fb37d3ff admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:29.690041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:29.690830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:29.691110 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:29.691379 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] PATH_INFO: `/v3/projects/0baea7ab89de426eb8f1e101f3abaf36` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:29.692548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0baea7ab89de426eb8f1e101f3abaf36)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:29.694487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:29.725302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:29.743925 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0baea7ab89de426eb8f1e101f3abaf36', 'request_id': 'req-565f6f24-d0ca-48cc-873c-8f7690f6a455'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.751084 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0baea7ab89de426eb8f1e101f3abaf36', 'request_id': 'req-565f6f24-d0ca-48cc-873c-8f7690f6a455'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.757460 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0baea7ab89de426eb8f1e101f3abaf36', 'request_id': 'req-565f6f24-d0ca-48cc-873c-8f7690f6a455'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.768560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] The token cache is being invalidate because project 0baea7ab89de426eb8f1e101f3abaf36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:30:29.768846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.769529 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-565f6f24-d0ca-48cc-873c-8f7690f6a455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:29.771125 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1398/2804] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:30:29 2022] DELETE /identity/v3/projects/0baea7ab89de426eb8f1e101f3abaf36 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:29.776069 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb3b44-ba92-4dc4-bf6b-602ae5cf0047 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:29.776534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb3b44-ba92-4dc4-bf6b-602ae5cf0047 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:29.776639 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb3b44-ba92-4dc4-bf6b-602ae5cf0047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:29.796855 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-6ccb3b44-ba92-4dc4-bf6b-602ae5cf0047 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:29.798092 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6ccb3b44-ba92-4dc4-bf6b-602ae5cf0047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:29.860988 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1407/2805] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:29.868824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3d0e4-0727-4929-bf03-ae17609351ad None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:29.869115 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3d0e4-0727-4929-bf03-ae17609351ad None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:29.869422 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3d0e4-0727-4929-bf03-ae17609351ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:29.890597 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f9f3d0e4-0727-4929-bf03-ae17609351ad None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:29.891900 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f9f3d0e4-0727-4929-bf03-ae17609351ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:29.900233 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1399/2806] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:29.904718 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb3b44-ba92-4dc4-bf6b-602ae5cf0047 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:29.914680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb3b44-ba92-4dc4-bf6b-602ae5cf0047 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:29.919361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225a97e-fddc-4487-bc90-9e507477abc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:29.920189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225a97e-fddc-4487-bc90-9e507477abc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:29.920470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225a97e-fddc-4487-bc90-9e507477abc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:29.920743 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225a97e-fddc-4487-bc90-9e507477abc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:29.921537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225a97e-fddc-4487-bc90-9e507477abc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:30:29.922111 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225a97e-fddc-4487-bc90-9e507477abc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:29.924096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225a97e-fddc-4487-bc90-9e507477abc1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:29.935621 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1408/2807] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:30:29.940458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3d0e4-0727-4929-bf03-ae17609351ad None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:29.950139 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3d0e4-0727-4929-bf03-ae17609351ad None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:29.955271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83e5db-3e4d-44df-a87b-8f8381ef2099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:29.955930 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83e5db-3e4d-44df-a87b-8f8381ef2099 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:29.956216 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83e5db-3e4d-44df-a87b-8f8381ef2099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:29.956494 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83e5db-3e4d-44df-a87b-8f8381ef2099 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:29.957209 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83e5db-3e4d-44df-a87b-8f8381ef2099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:29.959178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83e5db-3e4d-44df-a87b-8f8381ef2099 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:29.987194 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1400/2808] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:30:29 2022] POST /identity/v3/projects => generated 407 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:30:29.991173 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225a97e-fddc-4487-bc90-9e507477abc1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:30.001950 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225a97e-fddc-4487-bc90-9e507477abc1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:30.028326 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:30.061012 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61085e8d-98cf-4ae9-ad36-e0dd6eb6ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:30.061812 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61085e8d-98cf-4ae9-ad36-e0dd6eb6ca64 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.062201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61085e8d-98cf-4ae9-ad36-e0dd6eb6ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.062679 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61085e8d-98cf-4ae9-ad36-e0dd6eb6ca64 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.063490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61085e8d-98cf-4ae9-ad36-e0dd6eb6ca64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:30.065608 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61085e8d-98cf-4ae9-ad36-e0dd6eb6ca64 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:30.089312 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1409/2809] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:30:29 2022] POST /identity/v3/users => generated 304 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:30:30.090612 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83e5db-3e4d-44df-a87b-8f8381ef2099 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:30.101334 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83e5db-3e4d-44df-a87b-8f8381ef2099 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:30.106054 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590808e8-682a-4fd3-9cb8-5305c173934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:30.106718 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590808e8-682a-4fd3-9cb8-5305c173934f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.107006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590808e8-682a-4fd3-9cb8-5305c173934f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.107288 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590808e8-682a-4fd3-9cb8-5305c173934f admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.108023 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590808e8-682a-4fd3-9cb8-5305c173934f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:30.109711 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590808e8-682a-4fd3-9cb8-5305c173934f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:30.121512 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1401/2810] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:30:30.125470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61085e8d-98cf-4ae9-ad36-e0dd6eb6ca64 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:30.136368 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61085e8d-98cf-4ae9-ad36-e0dd6eb6ca64 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:30.141523 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac7f1a-bbbb-4b40-933d-8b04a2de0864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:30.142254 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac7f1a-bbbb-4b40-933d-8b04a2de0864 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.142555 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac7f1a-bbbb-4b40-933d-8b04a2de0864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.142838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac7f1a-bbbb-4b40-933d-8b04a2de0864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.143570 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac7f1a-bbbb-4b40-933d-8b04a2de0864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:30.145189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac7f1a-bbbb-4b40-933d-8b04a2de0864 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:30.156140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1410/2811] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:30:30.160094 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590808e8-682a-4fd3-9cb8-5305c173934f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:30.170374 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590808e8-682a-4fd3-9cb8-5305c173934f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:30.175056 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cdacc-d0f8-4795-8445-4af928f32e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:30.175942 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cdacc-d0f8-4795-8445-4af928f32e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.176226 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cdacc-d0f8-4795-8445-4af928f32e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.176503 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cdacc-d0f8-4795-8445-4af928f32e80 admin admin] PATH_INFO: `/v3/projects/acacb67e64254625a48fbab540b0a165/users/35aa36e3756447d8bf70ebc3ee94083f/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.192166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cdacc-d0f8-4795-8445-4af928f32e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acacb67e64254625a48fbab540b0a165, user_id=35aa36e3756447d8bf70ebc3ee94083f, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:30.194815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cdacc-d0f8-4795-8445-4af928f32e80 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:30.209144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1402/2812] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:30:30 2022] PUT /identity/v3/projects/acacb67e64254625a48fbab540b0a165/users/35aa36e3756447d8bf70ebc3ee94083f/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:30.219436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84596c87-9620-4bf8-a8be-c764d23ca876 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.219693 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84596c87-9620-4bf8-a8be-c764d23ca876 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.219975 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84596c87-9620-4bf8-a8be-c764d23ca876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.252861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-84596c87-9620-4bf8-a8be-c764d23ca876 None None] MFA Rules not processed for user `35aa36e3756447d8bf70ebc3ee94083f`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:30.254163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-84596c87-9620-4bf8-a8be-c764d23ca876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:30.322787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1411/2813] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:30 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:30.328204 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cdacc-d0f8-4795-8445-4af928f32e80 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:30.405815 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cdacc-d0f8-4795-8445-4af928f32e80 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:30.410944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff59d8-dc9b-40ec-b2ce-5e7c1286ad37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:30.411656 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff59d8-dc9b-40ec-b2ce-5e7c1286ad37 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.411951 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff59d8-dc9b-40ec-b2ce-5e7c1286ad37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.412224 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff59d8-dc9b-40ec-b2ce-5e7c1286ad37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.412941 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff59d8-dc9b-40ec-b2ce-5e7c1286ad37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:30.414912 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff59d8-dc9b-40ec-b2ce-5e7c1286ad37 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:30.450006 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1403/2814] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:30:30 2022] POST /identity/v3/projects => generated 409 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:30:30.454166 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84596c87-9620-4bf8-a8be-c764d23ca876 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:30.466322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84596c87-9620-4bf8-a8be-c764d23ca876 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:30.525025 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80ce6c-a071-4af4-8f6b-30327411b7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:30.525708 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80ce6c-a071-4af4-8f6b-30327411b7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.526029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80ce6c-a071-4af4-8f6b-30327411b7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.526416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80ce6c-a071-4af4-8f6b-30327411b7c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.527134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80ce6c-a071-4af4-8f6b-30327411b7c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:30.529093 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80ce6c-a071-4af4-8f6b-30327411b7c3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:30.551408 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1412/2815] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:30:30 2022] POST /identity/v3/users => generated 311 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:30:30.556873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff59d8-dc9b-40ec-b2ce-5e7c1286ad37 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:30.568530 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff59d8-dc9b-40ec-b2ce-5e7c1286ad37 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:30.574844 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2895a28-3f43-4fb4-a21a-5912e1b4449e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:30.575519 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2895a28-3f43-4fb4-a21a-5912e1b4449e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.575800 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2895a28-3f43-4fb4-a21a-5912e1b4449e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.576069 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2895a28-3f43-4fb4-a21a-5912e1b4449e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.576802 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2895a28-3f43-4fb4-a21a-5912e1b4449e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:30.578726 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2895a28-3f43-4fb4-a21a-5912e1b4449e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:30.590383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1404/2816] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:30:30.595081 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80ce6c-a071-4af4-8f6b-30327411b7c3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:30.606133 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80ce6c-a071-4af4-8f6b-30327411b7c3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:30.611611 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f80e4-b401-47b8-b62d-27df1388d277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:30.612549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f80e4-b401-47b8-b62d-27df1388d277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.612805 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f80e4-b401-47b8-b62d-27df1388d277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.613074 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f80e4-b401-47b8-b62d-27df1388d277 admin admin] PATH_INFO: `/v3/projects/96734a0ff4fc4401a2be9a3f839a6fdc/users/2d41a13b6b2649e18cdce0dbc588f5d4/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.629944 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f80e4-b401-47b8-b62d-27df1388d277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96734a0ff4fc4401a2be9a3f839a6fdc, user_id=2d41a13b6b2649e18cdce0dbc588f5d4, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:30.632640 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f80e4-b401-47b8-b62d-27df1388d277 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:30.649488 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1413/2817] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:30:30 2022] PUT /identity/v3/projects/96734a0ff4fc4401a2be9a3f839a6fdc/users/2d41a13b6b2649e18cdce0dbc588f5d4/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:30:30.653631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2895a28-3f43-4fb4-a21a-5912e1b4449e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:30.664459 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2895a28-3f43-4fb4-a21a-5912e1b4449e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:30.685353 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:30.724416 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c9f1b-e0cb-463b-b08d-a561636d81cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:30.725019 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c9f1b-e0cb-463b-b08d-a561636d81cf admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.725301 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c9f1b-e0cb-463b-b08d-a561636d81cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.725573 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c9f1b-e0cb-463b-b08d-a561636d81cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.726365 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c9f1b-e0cb-463b-b08d-a561636d81cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:30.727960 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c9f1b-e0cb-463b-b08d-a561636d81cf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:30.738487 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1405/2818] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:30:30.742434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f80e4-b401-47b8-b62d-27df1388d277 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:30.748188 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:30.755537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f80e4-b401-47b8-b62d-27df1388d277 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:30.760787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ebbb2-13c0-432e-831e-2ce09d3e7f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:30.761689 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ebbb2-13c0-432e-831e-2ce09d3e7f51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.762008 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ebbb2-13c0-432e-831e-2ce09d3e7f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.762277 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ebbb2-13c0-432e-831e-2ce09d3e7f51 admin admin] PATH_INFO: `/v3/projects/96734a0ff4fc4401a2be9a3f839a6fdc/users/2d41a13b6b2649e18cdce0dbc588f5d4/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.764322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ebbb2-13c0-432e-831e-2ce09d3e7f51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96734a0ff4fc4401a2be9a3f839a6fdc, user_id=2d41a13b6b2649e18cdce0dbc588f5d4, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:30.767357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ebbb2-13c0-432e-831e-2ce09d3e7f51 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:30.778490 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1414/2819] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:30:30 2022] PUT /identity/v3/projects/96734a0ff4fc4401a2be9a3f839a6fdc/users/2d41a13b6b2649e18cdce0dbc588f5d4/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:30.789975 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82ab3-2963-4f77-a69d-4b78bf108694 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.790266 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82ab3-2963-4f77-a69d-4b78bf108694 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.790536 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82ab3-2963-4f77-a69d-4b78bf108694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.826082 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-29c82ab3-2963-4f77-a69d-4b78bf108694 None None] MFA Rules not processed for user `2d41a13b6b2649e18cdce0dbc588f5d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:30.827342 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-29c82ab3-2963-4f77-a69d-4b78bf108694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:30.900024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1406/2820] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:30 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:30:30.911247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ebbb2-13c0-432e-831e-2ce09d3e7f51 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:30.913359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-a03ebbb2-13c0-432e-831e-2ce09d3e7f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:30.974272 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ebbb2-13c0-432e-831e-2ce09d3e7f51 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:30.979055 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d43f5-db4b-49eb-bec6-b6270372cea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:30.979658 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d43f5-db4b-49eb-bec6-b6270372cea1 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:30.979941 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d43f5-db4b-49eb-bec6-b6270372cea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:30.980214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d43f5-db4b-49eb-bec6-b6270372cea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:30.990050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d43f5-db4b-49eb-bec6-b6270372cea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:30.991543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d43f5-db4b-49eb-bec6-b6270372cea1 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:31.001645 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1415/2821] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:30 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:31.170593 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c82ab3-2963-4f77-a69d-4b78bf108694 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:31.181085 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c82ab3-2963-4f77-a69d-4b78bf108694 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:31.185032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c81a79-a4e2-4bfa-8657-8fc17b37a74d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:31.185703 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c81a79-a4e2-4bfa-8657-8fc17b37a74d service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:31.186021 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c81a79-a4e2-4bfa-8657-8fc17b37a74d service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:31.186300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c81a79-a4e2-4bfa-8657-8fc17b37a74d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:31.245137 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c81a79-a4e2-4bfa-8657-8fc17b37a74d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:31.246542 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c81a79-a4e2-4bfa-8657-8fc17b37a74d service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:31.255605 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1407/2822] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:31 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:38.656744 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d43f5-db4b-49eb-bec6-b6270372cea1 service cinder] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:38.720328 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d43f5-db4b-49eb-bec6-b6270372cea1 service cinder] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:38.726174 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:38.727062 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:38.727344 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:38.727642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] PATH_INFO: `/v3/users/35aa36e3756447d8bf70ebc3ee94083f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:38.728934 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=35aa36e3756447d8bf70ebc3ee94083f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:38.730953 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:38.782020 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35aa36e3756447d8bf70ebc3ee94083f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:38.782659 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35aa36e3756447d8bf70ebc3ee94083f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:38.791327 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35aa36e3756447d8bf70ebc3ee94083f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:38.798617 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35aa36e3756447d8bf70ebc3ee94083f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:38.799264 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35aa36e3756447d8bf70ebc3ee94083f'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:38.826258 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1416/2823] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:30:38 2022] DELETE /identity/v3/users/35aa36e3756447d8bf70ebc3ee94083f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:38.830996 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a9052-929f-40b2-b791-06c22053a4ee None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:38.831551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a9052-929f-40b2-b791-06c22053a4ee None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:38.831551 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a9052-929f-40b2-b791-06c22053a4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:38.850881 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-931a9052-929f-40b2-b791-06c22053a4ee None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:38.852136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-931a9052-929f-40b2-b791-06c22053a4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:38.893600 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:38.914719 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1408/2824] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:38.924616 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:38.926124 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:38.940037 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:38.997500 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22cf8b-f822-4ee4-89d2-145fd576fb0b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:39.001315 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d96a4a-4732-42f1-8923-7fb143786fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:39.001992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d96a4a-4732-42f1-8923-7fb143786fef service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:39.002272 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d96a4a-4732-42f1-8923-7fb143786fef service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:39.002545 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d96a4a-4732-42f1-8923-7fb143786fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:39.012484 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d96a4a-4732-42f1-8923-7fb143786fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:39.014190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d96a4a-4732-42f1-8923-7fb143786fef service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:39.025480 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1417/2825] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:39.303166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a9052-929f-40b2-b791-06c22053a4ee None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:39.304662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-931a9052-929f-40b2-b791-06c22053a4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:39.317323 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a9052-929f-40b2-b791-06c22053a4ee None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:39.323976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:39.324909 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:39.325208 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:39.325481 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] PATH_INFO: `/v3/projects/acacb67e64254625a48fbab540b0a165` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:39.327510 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acacb67e64254625a48fbab540b0a165)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:39.329675 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:39.380886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acacb67e64254625a48fbab540b0a165', 'request_id': 'req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:39.390720 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acacb67e64254625a48fbab540b0a165', 'request_id': 'req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:39.398294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acacb67e64254625a48fbab540b0a165', 'request_id': 'req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:39.410322 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] The token cache is being invalidate because project acacb67e64254625a48fbab540b0a165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:30:39.410611 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:39.411405 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:39.413144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1409/2826] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:30:39 2022] DELETE /identity/v3/projects/acacb67e64254625a48fbab540b0a165 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:30:39.417427 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d96a4a-4732-42f1-8923-7fb143786fef service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:39.428229 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d96a4a-4732-42f1-8923-7fb143786fef service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:39.486573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6834d-cf5b-49bb-ae0c-482311fa01b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:39.487446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6834d-cf5b-49bb-ae0c-482311fa01b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:39.487726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6834d-cf5b-49bb-ae0c-482311fa01b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:39.488003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6834d-cf5b-49bb-ae0c-482311fa01b8 admin admin] PATH_INFO: `/v3/users/2d41a13b6b2649e18cdce0dbc588f5d4` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:39.489352 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6834d-cf5b-49bb-ae0c-482311fa01b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d41a13b6b2649e18cdce0dbc588f5d4)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:39.491424 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6834d-cf5b-49bb-ae0c-482311fa01b8 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:39.541402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7f6834d-cf5b-49bb-ae0c-482311fa01b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d41a13b6b2649e18cdce0dbc588f5d4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:39.542110 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7f6834d-cf5b-49bb-ae0c-482311fa01b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d41a13b6b2649e18cdce0dbc588f5d4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:39.550843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7f6834d-cf5b-49bb-ae0c-482311fa01b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d41a13b6b2649e18cdce0dbc588f5d4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:39.558131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7f6834d-cf5b-49bb-ae0c-482311fa01b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d41a13b6b2649e18cdce0dbc588f5d4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:39.558756 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-b7f6834d-cf5b-49bb-ae0c-482311fa01b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d41a13b6b2649e18cdce0dbc588f5d4'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:39.577153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1418/2827] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:30:39 2022] DELETE /identity/v3/users/2d41a13b6b2649e18cdce0dbc588f5d4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:39.868151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:39.869696 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:39.883665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa5f0af-f9bf-4378-80c0-bb25d34f17c2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:39.944335 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:39.945237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:39.945514 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:39.945803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] PATH_INFO: `/v3/projects/96734a0ff4fc4401a2be9a3f839a6fdc` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:39.947104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=96734a0ff4fc4401a2be9a3f839a6fdc)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:39.949036 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:40.003882 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96734a0ff4fc4401a2be9a3f839a6fdc', 'request_id': 'req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:40.013867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96734a0ff4fc4401a2be9a3f839a6fdc', 'request_id': 'req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:40.021101 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96734a0ff4fc4401a2be9a3f839a6fdc', 'request_id': 'req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:40.027670 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:40.033169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] The token cache is being invalidate because project 96734a0ff4fc4401a2be9a3f839a6fdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:30:40.033552 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:40.034338 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-dfd31682-ad74-4c09-8fcc-d0d97932e6ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:40.036317 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1410/2828] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:30:39 2022] DELETE /identity/v3/projects/96734a0ff4fc4401a2be9a3f839a6fdc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:40.041898 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b0eba-4772-434e-87fb-34226dbbd0d8 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.042390 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b0eba-4772-434e-87fb-34226dbbd0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.042508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b0eba-4772-434e-87fb-34226dbbd0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.061361 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-cd1b0eba-4772-434e-87fb-34226dbbd0d8 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:40.062539 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-cd1b0eba-4772-434e-87fb-34226dbbd0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:40.093016 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:40.127527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1419/2829] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:40.135992 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa1888-a10f-40c8-8e70-09b5e9436a07 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.136281 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa1888-a10f-40c8-8e70-09b5e9436a07 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.136559 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa1888-a10f-40c8-8e70-09b5e9436a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.156993 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8bfa1888-a10f-40c8-8e70-09b5e9436a07 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:40.158228 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8bfa1888-a10f-40c8-8e70-09b5e9436a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:40.166489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1411/2830] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:40.170807 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b0eba-4772-434e-87fb-34226dbbd0d8 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:40.180760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b0eba-4772-434e-87fb-34226dbbd0d8 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:40.185770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c45af-ef17-4870-b9ef-11440bec44ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:40.186482 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c45af-ef17-4870-b9ef-11440bec44ab admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.186752 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c45af-ef17-4870-b9ef-11440bec44ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.187032 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c45af-ef17-4870-b9ef-11440bec44ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.187760 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c45af-ef17-4870-b9ef-11440bec44ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80283) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:30:40.188064 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c45af-ef17-4870-b9ef-11440bec44ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:40.189653 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c45af-ef17-4870-b9ef-11440bec44ab admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:40.201636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1420/2831] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:30:40.205918 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa1888-a10f-40c8-8e70-09b5e9436a07 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:40.215860 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa1888-a10f-40c8-8e70-09b5e9436a07 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:40.221462 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21f50f-700f-496e-981c-8968d4b1ecd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:40.222155 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21f50f-700f-496e-981c-8968d4b1ecd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.222420 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21f50f-700f-496e-981c-8968d4b1ecd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.222730 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21f50f-700f-496e-981c-8968d4b1ecd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.223533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21f50f-700f-496e-981c-8968d4b1ecd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:40.225473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21f50f-700f-496e-981c-8968d4b1ecd9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:40.255609 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1412/2832] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:30:40 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:30:40.260104 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c45af-ef17-4870-b9ef-11440bec44ab admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:40.270771 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c45af-ef17-4870-b9ef-11440bec44ab admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:40.327563 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85630df6-9874-4500-bddb-f10877abdbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:40.328203 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85630df6-9874-4500-bddb-f10877abdbd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.328492 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85630df6-9874-4500-bddb-f10877abdbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.328770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85630df6-9874-4500-bddb-f10877abdbd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.329558 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85630df6-9874-4500-bddb-f10877abdbd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:40.331671 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85630df6-9874-4500-bddb-f10877abdbd1 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:40.356029 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1421/2833] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:30:40 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:30:40.360966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21f50f-700f-496e-981c-8968d4b1ecd9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:40.373144 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21f50f-700f-496e-981c-8968d4b1ecd9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:40.378024 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf962d50-2bc2-4f6c-ad75-6fd71d99ff29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:40.378694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf962d50-2bc2-4f6c-ad75-6fd71d99ff29 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.378979 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf962d50-2bc2-4f6c-ad75-6fd71d99ff29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.379257 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf962d50-2bc2-4f6c-ad75-6fd71d99ff29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.380001 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf962d50-2bc2-4f6c-ad75-6fd71d99ff29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:40.381640 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf962d50-2bc2-4f6c-ad75-6fd71d99ff29 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:40.392780 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1413/2834] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:30:40.397030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85630df6-9874-4500-bddb-f10877abdbd1 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:40.407680 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85630df6-9874-4500-bddb-f10877abdbd1 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:40.413596 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef49301-706c-4dc7-815e-597f008ed217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:40.414357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef49301-706c-4dc7-815e-597f008ed217 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.414678 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef49301-706c-4dc7-815e-597f008ed217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.414990 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef49301-706c-4dc7-815e-597f008ed217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.415710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef49301-706c-4dc7-815e-597f008ed217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:40.417322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef49301-706c-4dc7-815e-597f008ed217 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:40.429765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1422/2835] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:30:40.434547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf962d50-2bc2-4f6c-ad75-6fd71d99ff29 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:40.445660 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf962d50-2bc2-4f6c-ad75-6fd71d99ff29 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:40.451837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe06d38-f9ef-4c70-a84a-9aa31bef1556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:40.452223 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe06d38-f9ef-4c70-a84a-9aa31bef1556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.452547 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe06d38-f9ef-4c70-a84a-9aa31bef1556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.453028 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe06d38-f9ef-4c70-a84a-9aa31bef1556 admin admin] PATH_INFO: `/v3/projects/e4db648294b1471581e14c36036b3e2b/users/5e367a281de44807b1f33ef8165223ee/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.470289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe06d38-f9ef-4c70-a84a-9aa31bef1556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4db648294b1471581e14c36036b3e2b, user_id=5e367a281de44807b1f33ef8165223ee, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:40.472988 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe06d38-f9ef-4c70-a84a-9aa31bef1556 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:40.489111 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1414/2836] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:30:40 2022] PUT /identity/v3/projects/e4db648294b1471581e14c36036b3e2b/users/5e367a281de44807b1f33ef8165223ee/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:40.499536 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89bb4-8fc9-41c3-adc7-f189f64ffabf None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.499856 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89bb4-8fc9-41c3-adc7-f189f64ffabf None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.500134 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89bb4-8fc9-41c3-adc7-f189f64ffabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.536399 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-c9b89bb4-8fc9-41c3-adc7-f189f64ffabf None None] MFA Rules not processed for user `5e367a281de44807b1f33ef8165223ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:40.537797 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-c9b89bb4-8fc9-41c3-adc7-f189f64ffabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:40.613604 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1423/2837] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:40.618681 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe06d38-f9ef-4c70-a84a-9aa31bef1556 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:40.683828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe06d38-f9ef-4c70-a84a-9aa31bef1556 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:40.689771 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95917f-e166-412a-ae85-b2f65b5b1b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:40.690539 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95917f-e166-412a-ae85-b2f65b5b1b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.690821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95917f-e166-412a-ae85-b2f65b5b1b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.691092 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95917f-e166-412a-ae85-b2f65b5b1b89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.691846 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95917f-e166-412a-ae85-b2f65b5b1b89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:40.693805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95917f-e166-412a-ae85-b2f65b5b1b89 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:40.725027 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1415/2838] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:30:40 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:30:40.729434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89bb4-8fc9-41c3-adc7-f189f64ffabf None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:40.742784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89bb4-8fc9-41c3-adc7-f189f64ffabf None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:40.769471 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:40.806540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bb445-4dc9-4b12-81bb-ab86e14f5dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:40.807226 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bb445-4dc9-4b12-81bb-ab86e14f5dba admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.807508 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bb445-4dc9-4b12-81bb-ab86e14f5dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.807786 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bb445-4dc9-4b12-81bb-ab86e14f5dba admin admin] PATH_INFO: `/v3/users` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.808491 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bb445-4dc9-4b12-81bb-ab86e14f5dba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:40.810488 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bb445-4dc9-4b12-81bb-ab86e14f5dba admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:40.832885 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1424/2839] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:30:40 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:30:40.837833 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95917f-e166-412a-ae85-b2f65b5b1b89 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:40.848158 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95917f-e166-412a-ae85-b2f65b5b1b89 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:40.853189 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914cc11-03d1-41c0-920f-c80ceff81dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:40.853848 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cc11-03d1-41c0-920f-c80ceff81dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.854193 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cc11-03d1-41c0-920f-c80ceff81dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.854447 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cc11-03d1-41c0-920f-c80ceff81dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.855154 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914cc11-03d1-41c0-920f-c80ceff81dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:40.856767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914cc11-03d1-41c0-920f-c80ceff81dc1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:40.867390 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1416/2840] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:30:40.873303 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bb445-4dc9-4b12-81bb-ab86e14f5dba admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:40.884415 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bb445-4dc9-4b12-81bb-ab86e14f5dba admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:40.891003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da0acf-427e-431d-8b29-bf24cf0c7f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:40.892253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da0acf-427e-431d-8b29-bf24cf0c7f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:40.892253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da0acf-427e-431d-8b29-bf24cf0c7f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:40.892470 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da0acf-427e-431d-8b29-bf24cf0c7f12 admin admin] PATH_INFO: `/v3/projects/b7bb8952e28d4b9abb856e51ba6c6832/users/56b53ffef3b94c0ea3fd30b8f2228a3f/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:40.908895 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da0acf-427e-431d-8b29-bf24cf0c7f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7bb8952e28d4b9abb856e51ba6c6832, user_id=56b53ffef3b94c0ea3fd30b8f2228a3f, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:40.911770 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da0acf-427e-431d-8b29-bf24cf0c7f12 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:40.927112 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1425/2841] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:30:40 2022] PUT /identity/v3/projects/b7bb8952e28d4b9abb856e51ba6c6832/users/56b53ffef3b94c0ea3fd30b8f2228a3f/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:40.931219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914cc11-03d1-41c0-920f-c80ceff81dc1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:40.942091 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914cc11-03d1-41c0-920f-c80ceff81dc1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:41.006652 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf0984-a2ff-4c6b-bb6b-a1c65642c59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:41.007332 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf0984-a2ff-4c6b-bb6b-a1c65642c59d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:41.007613 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf0984-a2ff-4c6b-bb6b-a1c65642c59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:41.007890 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf0984-a2ff-4c6b-bb6b-a1c65642c59d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:41.008623 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf0984-a2ff-4c6b-bb6b-a1c65642c59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:41.010273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf0984-a2ff-4c6b-bb6b-a1c65642c59d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:41.020455 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1417/2842] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:30:41.027222 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da0acf-427e-431d-8b29-bf24cf0c7f12 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:41.041225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da0acf-427e-431d-8b29-bf24cf0c7f12 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:41.047087 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f61f9-aa0b-441d-bdac-1c8e4369d366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:41.047993 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f61f9-aa0b-441d-bdac-1c8e4369d366 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:41.048252 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f61f9-aa0b-441d-bdac-1c8e4369d366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:41.048526 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f61f9-aa0b-441d-bdac-1c8e4369d366 admin admin] PATH_INFO: `/v3/projects/b7bb8952e28d4b9abb856e51ba6c6832/users/56b53ffef3b94c0ea3fd30b8f2228a3f/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:41.050541 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f61f9-aa0b-441d-bdac-1c8e4369d366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7bb8952e28d4b9abb856e51ba6c6832, user_id=56b53ffef3b94c0ea3fd30b8f2228a3f, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:41.053574 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f61f9-aa0b-441d-bdac-1c8e4369d366 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:41.065737 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1426/2843] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:30:41 2022] PUT /identity/v3/projects/b7bb8952e28d4b9abb856e51ba6c6832/users/56b53ffef3b94c0ea3fd30b8f2228a3f/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:41.078616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90cbc3-a873-4593-b8d9-01bd7c3e85a8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:41.078949 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90cbc3-a873-4593-b8d9-01bd7c3e85a8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:41.079307 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90cbc3-a873-4593-b8d9-01bd7c3e85a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:41.090767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:41.113079 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4d90cbc3-a873-4593-b8d9-01bd7c3e85a8 None None] MFA Rules not processed for user `56b53ffef3b94c0ea3fd30b8f2228a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:41.114483 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4d90cbc3-a873-4593-b8d9-01bd7c3e85a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:41.190616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1418/2844] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:30:41.201720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f61f9-aa0b-441d-bdac-1c8e4369d366 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:41.203838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-779f61f9-aa0b-441d-bdac-1c8e4369d366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:41.265234 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f61f9-aa0b-441d-bdac-1c8e4369d366 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:41.269710 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189fd61-1bff-4be5-943d-491cba86ceaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:41.270462 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189fd61-1bff-4be5-943d-491cba86ceaf service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:41.270747 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189fd61-1bff-4be5-943d-491cba86ceaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:41.271023 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189fd61-1bff-4be5-943d-491cba86ceaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:41.281128 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189fd61-1bff-4be5-943d-491cba86ceaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:41.282623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189fd61-1bff-4be5-943d-491cba86ceaf service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:41.292700 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1427/2845] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:41.874558 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90cbc3-a873-4593-b8d9-01bd7c3e85a8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:41.939500 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90cbc3-a873-4593-b8d9-01bd7c3e85a8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:41.945546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:41.946489 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:41.946772 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:41.947042 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] PATH_INFO: `/v3/users/5e367a281de44807b1f33ef8165223ee` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:41.948400 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e367a281de44807b1f33ef8165223ee)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:41.950358 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:41.999134 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e367a281de44807b1f33ef8165223ee'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:41.999839 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e367a281de44807b1f33ef8165223ee'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.000516 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e367a281de44807b1f33ef8165223ee'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.013299 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e367a281de44807b1f33ef8165223ee'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.022798 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e367a281de44807b1f33ef8165223ee'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.041229 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1419/2846] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:30:41 2022] DELETE /identity/v3/users/5e367a281de44807b1f33ef8165223ee => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:42.049875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ffcf7-a3f2-4bff-a959-5271842a0c17 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:42.050436 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ffcf7-a3f2-4bff-a959-5271842a0c17 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:42.050540 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ffcf7-a3f2-4bff-a959-5271842a0c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:42.075291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-b39ffcf7-a3f2-4bff-a959-5271842a0c17 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:42.076736 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b39ffcf7-a3f2-4bff-a959-5271842a0c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:42.153866 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1428/2847] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:42.161626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:42.163318 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:42.175104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:42.222934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f013cd0-0d8f-47d4-84a1-ae27edc775f1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:42.227138 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a1f5b-2c6d-4168-ae72-0cdbe9f03c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:42.227786 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a1f5b-2c6d-4168-ae72-0cdbe9f03c02 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:42.228067 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a1f5b-2c6d-4168-ae72-0cdbe9f03c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:42.228367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a1f5b-2c6d-4168-ae72-0cdbe9f03c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:42.238093 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a1f5b-2c6d-4168-ae72-0cdbe9f03c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:42.239532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a1f5b-2c6d-4168-ae72-0cdbe9f03c02 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:42.248873 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1420/2848] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:42.539847 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ffcf7-a3f2-4bff-a959-5271842a0c17 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:42.541485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-b39ffcf7-a3f2-4bff-a959-5271842a0c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:42.557729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ffcf7-a3f2-4bff-a959-5271842a0c17 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:42.563838 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:42.564796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:42.565050 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:42.565384 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] PATH_INFO: `/v3/projects/e4db648294b1471581e14c36036b3e2b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:42.566713 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4db648294b1471581e14c36036b3e2b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:42.568715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:42.593131 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:42.628547 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4db648294b1471581e14c36036b3e2b', 'request_id': 'req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.640340 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4db648294b1471581e14c36036b3e2b', 'request_id': 'req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.647402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4db648294b1471581e14c36036b3e2b', 'request_id': 'req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.657951 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] The token cache is being invalidate because project e4db648294b1471581e14c36036b3e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:30:42.658244 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.658917 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.660534 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1429/2849] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:30:42 2022] DELETE /identity/v3/projects/e4db648294b1471581e14c36036b3e2b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:42.664747 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a1f5b-2c6d-4168-ae72-0cdbe9f03c02 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:42.674622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a1f5b-2c6d-4168-ae72-0cdbe9f03c02 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:42.751172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7059b7c-5941-44ab-9429-2d934e8bfaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:42.752478 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7059b7c-5941-44ab-9429-2d934e8bfaa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:42.752806 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7059b7c-5941-44ab-9429-2d934e8bfaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:42.753252 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7059b7c-5941-44ab-9429-2d934e8bfaa1 admin admin] PATH_INFO: `/v3/users/56b53ffef3b94c0ea3fd30b8f2228a3f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:42.754737 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7059b7c-5941-44ab-9429-2d934e8bfaa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b53ffef3b94c0ea3fd30b8f2228a3f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:42.757659 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7059b7c-5941-44ab-9429-2d934e8bfaa1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:42.815850 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f7059b7c-5941-44ab-9429-2d934e8bfaa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b53ffef3b94c0ea3fd30b8f2228a3f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.816532 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f7059b7c-5941-44ab-9429-2d934e8bfaa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b53ffef3b94c0ea3fd30b8f2228a3f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.817151 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f7059b7c-5941-44ab-9429-2d934e8bfaa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b53ffef3b94c0ea3fd30b8f2228a3f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.831314 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f7059b7c-5941-44ab-9429-2d934e8bfaa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b53ffef3b94c0ea3fd30b8f2228a3f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.843757 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f7059b7c-5941-44ab-9429-2d934e8bfaa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b53ffef3b94c0ea3fd30b8f2228a3f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:42.864674 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1421/2850] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:30:42 2022] DELETE /identity/v3/users/56b53ffef3b94c0ea3fd30b8f2228a3f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:43.216742 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:43.218593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:43.236879 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d653cb-88fe-42da-b0bd-3e697c3ec9fb admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:43.315985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:43.316784 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:43.319899 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:43.320192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] PATH_INFO: `/v3/projects/b7bb8952e28d4b9abb856e51ba6c6832` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:43.321520 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7bb8952e28d4b9abb856e51ba6c6832)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:43.323493 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:43.387459 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7bb8952e28d4b9abb856e51ba6c6832', 'request_id': 'req-ff8b322e-63dd-4edc-a065-52fe8853f622'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:43.398688 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7bb8952e28d4b9abb856e51ba6c6832', 'request_id': 'req-ff8b322e-63dd-4edc-a065-52fe8853f622'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:43.405636 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7bb8952e28d4b9abb856e51ba6c6832', 'request_id': 'req-ff8b322e-63dd-4edc-a065-52fe8853f622'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:43.416120 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] The token cache is being invalidate because project b7bb8952e28d4b9abb856e51ba6c6832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:30:43.416405 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:43.417044 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ff8b322e-63dd-4edc-a065-52fe8853f622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:43.419250 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1430/2851] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:30:43 2022] DELETE /identity/v3/projects/b7bb8952e28d4b9abb856e51ba6c6832 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:43.424523 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c50ee-7569-4adf-9de0-7694b92ad64d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:43.425072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c50ee-7569-4adf-9de0-7694b92ad64d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:43.425072 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c50ee-7569-4adf-9de0-7694b92ad64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:43.450321 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-025c50ee-7569-4adf-9de0-7694b92ad64d None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:43.451571 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-025c50ee-7569-4adf-9de0-7694b92ad64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:43.482308 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:43.519727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1422/2852] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:43.528132 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a261f-db6f-41bf-b2a4-78f23cf67568 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:43.528380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a261f-db6f-41bf-b2a4-78f23cf67568 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:43.528707 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a261f-db6f-41bf-b2a4-78f23cf67568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:43.551391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-9e1a261f-db6f-41bf-b2a4-78f23cf67568 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:43.552593 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-9e1a261f-db6f-41bf-b2a4-78f23cf67568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:43.562320 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1431/2853] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:43.567039 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c50ee-7569-4adf-9de0-7694b92ad64d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:43.577305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c50ee-7569-4adf-9de0-7694b92ad64d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:43.583869 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f5d61-1b14-413b-b449-8e072e6d7a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:43.584622 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f5d61-1b14-413b-b449-8e072e6d7a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:43.584910 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f5d61-1b14-413b-b449-8e072e6d7a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:43.585185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f5d61-1b14-413b-b449-8e072e6d7a31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:43.585934 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f5d61-1b14-413b-b449-8e072e6d7a31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:30:43.586271 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f5d61-1b14-413b-b449-8e072e6d7a31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:43.587891 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f5d61-1b14-413b-b449-8e072e6d7a31 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:43.601480 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1423/2854] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:30:43.606505 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a261f-db6f-41bf-b2a4-78f23cf67568 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:43.616240 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a261f-db6f-41bf-b2a4-78f23cf67568 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:43.621861 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efdd99-9a27-4242-a9cd-cf5575f5ed18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:43.622543 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efdd99-9a27-4242-a9cd-cf5575f5ed18 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:43.622826 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efdd99-9a27-4242-a9cd-cf5575f5ed18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:43.623098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efdd99-9a27-4242-a9cd-cf5575f5ed18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:43.623791 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efdd99-9a27-4242-a9cd-cf5575f5ed18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:43.625701 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efdd99-9a27-4242-a9cd-cf5575f5ed18 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:43.655331 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1432/2855] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:30:43 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:30:43.659598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f5d61-1b14-413b-b449-8e072e6d7a31 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:43.670121 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f5d61-1b14-413b-b449-8e072e6d7a31 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:43.730553 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902ff5c-080d-4810-b02b-482e5ce20ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:43.731263 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902ff5c-080d-4810-b02b-482e5ce20ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:43.731549 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902ff5c-080d-4810-b02b-482e5ce20ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:43.731828 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902ff5c-080d-4810-b02b-482e5ce20ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:43.732560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902ff5c-080d-4810-b02b-482e5ce20ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:43.734546 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902ff5c-080d-4810-b02b-482e5ce20ca4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:43.758588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1424/2856] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:30:43 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 04 09:30:43.762777 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efdd99-9a27-4242-a9cd-cf5575f5ed18 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:43.774073 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efdd99-9a27-4242-a9cd-cf5575f5ed18 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:43.779190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d291c-ee7a-4ab5-9ad1-16dd9289a7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:43.780003 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d291c-ee7a-4ab5-9ad1-16dd9289a7be admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:43.780357 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d291c-ee7a-4ab5-9ad1-16dd9289a7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:43.780663 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d291c-ee7a-4ab5-9ad1-16dd9289a7be admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:43.781456 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d291c-ee7a-4ab5-9ad1-16dd9289a7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:43.783102 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d291c-ee7a-4ab5-9ad1-16dd9289a7be admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:43.785483 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:43.796832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1433/2857] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:30:43.801298 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902ff5c-080d-4810-b02b-482e5ce20ca4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:43.812330 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902ff5c-080d-4810-b02b-482e5ce20ca4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:43.818219 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d2ba1-f522-4137-9cda-2780d1221337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:43.818944 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d2ba1-f522-4137-9cda-2780d1221337 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:43.819373 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d2ba1-f522-4137-9cda-2780d1221337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:43.819667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d2ba1-f522-4137-9cda-2780d1221337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:43.820392 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d2ba1-f522-4137-9cda-2780d1221337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:43.822050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d2ba1-f522-4137-9cda-2780d1221337 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:43.832327 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1425/2858] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:30:43.836852 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d291c-ee7a-4ab5-9ad1-16dd9289a7be admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:43.848053 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d291c-ee7a-4ab5-9ad1-16dd9289a7be admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:43.853036 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0018c-3bf4-4621-8390-0fd5ef4495b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:43.853982 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0018c-3bf4-4621-8390-0fd5ef4495b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:43.854266 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0018c-3bf4-4621-8390-0fd5ef4495b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:43.854549 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0018c-3bf4-4621-8390-0fd5ef4495b3 admin admin] PATH_INFO: `/v3/projects/afe3d042da4d4515a807725bc6b4934b/users/444c7032a2014fcf97c3aa63b838b4cf/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:43.869191 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0018c-3bf4-4621-8390-0fd5ef4495b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe3d042da4d4515a807725bc6b4934b, user_id=444c7032a2014fcf97c3aa63b838b4cf, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:43.871906 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0018c-3bf4-4621-8390-0fd5ef4495b3 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:43.886726 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1434/2859] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:30:43 2022] PUT /identity/v3/projects/afe3d042da4d4515a807725bc6b4934b/users/444c7032a2014fcf97c3aa63b838b4cf/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:43.898548 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4b3e-19c1-4a8c-bc19-e22e529a6d46 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:43.898864 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4b3e-19c1-4a8c-bc19-e22e529a6d46 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:43.899143 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4b3e-19c1-4a8c-bc19-e22e529a6d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:43.927098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-36ea4b3e-19c1-4a8c-bc19-e22e529a6d46 None None] MFA Rules not processed for user `444c7032a2014fcf97c3aa63b838b4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:43.928367 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-36ea4b3e-19c1-4a8c-bc19-e22e529a6d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:43.997435 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1426/2860] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:44.009280 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0018c-3bf4-4621-8390-0fd5ef4495b3 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:44.011332 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-91c0018c-3bf4-4621-8390-0fd5ef4495b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:44.074717 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0018c-3bf4-4621-8390-0fd5ef4495b3 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:44.079380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f78e12-c8b3-42de-bf4d-2f5ab395e583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:44.080079 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f78e12-c8b3-42de-bf4d-2f5ab395e583 service cinder] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:44.080333 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f78e12-c8b3-42de-bf4d-2f5ab395e583 service cinder] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:44.080594 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f78e12-c8b3-42de-bf4d-2f5ab395e583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:44.090391 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f78e12-c8b3-42de-bf4d-2f5ab395e583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:44.091832 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f78e12-c8b3-42de-bf4d-2f5ab395e583 service cinder] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:44.100974 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1435/2861] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:44.120855 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea4b3e-19c1-4a8c-bc19-e22e529a6d46 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:44.153567 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:44.192576 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea4b3e-19c1-4a8c-bc19-e22e529a6d46 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:44.200626 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:44.202015 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:44.202417 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:44.202808 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] PATH_INFO: `/v3/users/444c7032a2014fcf97c3aa63b838b4cf` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:44.204858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=444c7032a2014fcf97c3aa63b838b4cf)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:44.207682 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:44.270041 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '444c7032a2014fcf97c3aa63b838b4cf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:44.270693 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '444c7032a2014fcf97c3aa63b838b4cf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:44.271300 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '444c7032a2014fcf97c3aa63b838b4cf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:44.280819 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '444c7032a2014fcf97c3aa63b838b4cf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:44.288388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '444c7032a2014fcf97c3aa63b838b4cf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:44.306767 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1427/2862] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:30:44 2022] DELETE /identity/v3/users/444c7032a2014fcf97c3aa63b838b4cf => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:44.311631 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2489f-52a1-4940-9eb4-6ed19025ae69 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:44.312168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2489f-52a1-4940-9eb4-6ed19025ae69 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:44.312168 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2489f-52a1-4940-9eb4-6ed19025ae69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:44.331963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-8fb2489f-52a1-4940-9eb4-6ed19025ae69 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:44.333167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8fb2489f-52a1-4940-9eb4-6ed19025ae69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:44.395647 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1436/2863] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:44.405863 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:44.407506 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:44.474087 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5411c87-d1fd-44b7-a9c2-df535b62c612 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:44.478412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dfaa8-15bd-4f45-8c1b-bd06364718e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:44.479063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dfaa8-15bd-4f45-8c1b-bd06364718e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:44.479346 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dfaa8-15bd-4f45-8c1b-bd06364718e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:44.479616 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dfaa8-15bd-4f45-8c1b-bd06364718e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:44.488908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dfaa8-15bd-4f45-8c1b-bd06364718e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:44.490384 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dfaa8-15bd-4f45-8c1b-bd06364718e3 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:44.500157 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1428/2864] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 04 09:30:44.777141 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2489f-52a1-4940-9eb4-6ed19025ae69 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:44.778955 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-8fb2489f-52a1-4940-9eb4-6ed19025ae69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:44.797225 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2489f-52a1-4940-9eb4-6ed19025ae69 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:44.802851 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:44.803823 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:44.804106 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:44.804380 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] PATH_INFO: `/v3/projects/afe3d042da4d4515a807725bc6b4934b` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:44.805774 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe3d042da4d4515a807725bc6b4934b)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:44.807750 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:44.858467 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe3d042da4d4515a807725bc6b4934b', 'request_id': 'req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:44.869144 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe3d042da4d4515a807725bc6b4934b', 'request_id': 'req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:44.876819 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe3d042da4d4515a807725bc6b4934b', 'request_id': 'req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:44.885625 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:44.890488 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] The token cache is being invalidate because project afe3d042da4d4515a807725bc6b4934b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:30:44.890800 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:44.892009 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-388cb5fa-5adb-4c5c-9cbe-f8a33a1c2180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:30:44.893814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1437/2865] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:30:44 2022] DELETE /identity/v3/projects/afe3d042da4d4515a807725bc6b4934b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:44.899382 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca9386-4b34-4d1f-976d-ca3f3777c019 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:44.899680 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca9386-4b34-4d1f-976d-ca3f3777c019 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:44.899961 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca9386-4b34-4d1f-976d-ca3f3777c019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:44.918915 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3aca9386-4b34-4d1f-976d-ca3f3777c019 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:44.920129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3aca9386-4b34-4d1f-976d-ca3f3777c019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:44.972259 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:44.990928 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1429/2866] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:30:44.999071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363118ab-3a30-4807-8b7f-860eb2ec6bc4 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:44.999364 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363118ab-3a30-4807-8b7f-860eb2ec6bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:44.999642 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363118ab-3a30-4807-8b7f-860eb2ec6bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:45.019585 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-363118ab-3a30-4807-8b7f-860eb2ec6bc4 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:45.020794 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-363118ab-3a30-4807-8b7f-860eb2ec6bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:45.030714 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1438/2867] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:45.035340 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca9386-4b34-4d1f-976d-ca3f3777c019 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:45.051687 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca9386-4b34-4d1f-976d-ca3f3777c019 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:45.058867 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe29ce-0449-47be-88e3-eec85bdbb911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:45.059550 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe29ce-0449-47be-88e3-eec85bdbb911 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:45.059837 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe29ce-0449-47be-88e3-eec85bdbb911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:45.060098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe29ce-0449-47be-88e3-eec85bdbb911 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:45.060861 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe29ce-0449-47be-88e3-eec85bdbb911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:30:45.061172 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe29ce-0449-47be-88e3-eec85bdbb911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:45.063824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe29ce-0449-47be-88e3-eec85bdbb911 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:45.077518 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1430/2868] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:30:45.082055 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363118ab-3a30-4807-8b7f-860eb2ec6bc4 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:45.091600 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363118ab-3a30-4807-8b7f-860eb2ec6bc4 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:45.096692 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c6b7f-e45c-48ea-893f-b06e381ccb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:45.097433 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c6b7f-e45c-48ea-893f-b06e381ccb12 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:45.097787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c6b7f-e45c-48ea-893f-b06e381ccb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:45.098163 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c6b7f-e45c-48ea-893f-b06e381ccb12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:45.098954 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c6b7f-e45c-48ea-893f-b06e381ccb12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:45.101755 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c6b7f-e45c-48ea-893f-b06e381ccb12 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:45.135190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1439/2869] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:30:45 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:30:45.139599 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe29ce-0449-47be-88e3-eec85bdbb911 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:45.151669 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe29ce-0449-47be-88e3-eec85bdbb911 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:45.210052 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04566c5c-5003-408e-bddd-4a6c8b6ee882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:45.210715 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04566c5c-5003-408e-bddd-4a6c8b6ee882 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:45.210997 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04566c5c-5003-408e-bddd-4a6c8b6ee882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:45.211273 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04566c5c-5003-408e-bddd-4a6c8b6ee882 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:45.211971 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04566c5c-5003-408e-bddd-4a6c8b6ee882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:45.213990 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04566c5c-5003-408e-bddd-4a6c8b6ee882 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:45.236297 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1431/2870] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:30:45 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:30:45.240979 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c6b7f-e45c-48ea-893f-b06e381ccb12 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:45.252498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c6b7f-e45c-48ea-893f-b06e381ccb12 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:45.259065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933b995-b0ff-48e4-9776-662f8c4df0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:45.259985 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933b995-b0ff-48e4-9776-662f8c4df0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:45.260287 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933b995-b0ff-48e4-9776-662f8c4df0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:45.260560 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933b995-b0ff-48e4-9776-662f8c4df0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:45.261317 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933b995-b0ff-48e4-9776-662f8c4df0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:45.262997 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933b995-b0ff-48e4-9776-662f8c4df0c2 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:45.275358 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1440/2871] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:30:45.279631 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04566c5c-5003-408e-bddd-4a6c8b6ee882 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:45.290988 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04566c5c-5003-408e-bddd-4a6c8b6ee882 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:45.296728 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cdc02-ece8-4c4e-8009-5369b7714542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:45.297485 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cdc02-ece8-4c4e-8009-5369b7714542 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:45.297954 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cdc02-ece8-4c4e-8009-5369b7714542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:45.298237 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cdc02-ece8-4c4e-8009-5369b7714542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:45.298977 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cdc02-ece8-4c4e-8009-5369b7714542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:45.300658 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cdc02-ece8-4c4e-8009-5369b7714542 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:45.311895 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1432/2872] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:30:45.315994 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933b995-b0ff-48e4-9776-662f8c4df0c2 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:45.326571 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933b995-b0ff-48e4-9776-662f8c4df0c2 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:45.331881 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746bff-c84e-4f94-88e9-07bbbd675a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:45.332782 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746bff-c84e-4f94-88e9-07bbbd675a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:45.333065 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746bff-c84e-4f94-88e9-07bbbd675a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:45.333342 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746bff-c84e-4f94-88e9-07bbbd675a9b admin admin] PATH_INFO: `/v3/projects/f4be24e33fe0477b8fb0caf9541d8d1a/users/d08d24e1e0164a68843862a56973c830/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:45.349306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8746bff-c84e-4f94-88e9-07bbbd675a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4be24e33fe0477b8fb0caf9541d8d1a, user_id=d08d24e1e0164a68843862a56973c830, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:45.352285 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8746bff-c84e-4f94-88e9-07bbbd675a9b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:45.367346 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1441/2873] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:30:45 2022] PUT /identity/v3/projects/f4be24e33fe0477b8fb0caf9541d8d1a/users/d08d24e1e0164a68843862a56973c830/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:30:45.377858 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66ce7c-fec4-4082-af1b-d8c13e04096b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:45.378183 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66ce7c-fec4-4082-af1b-d8c13e04096b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:45.378463 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66ce7c-fec4-4082-af1b-d8c13e04096b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:45.408451 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6d66ce7c-fec4-4082-af1b-d8c13e04096b None None] MFA Rules not processed for user `d08d24e1e0164a68843862a56973c830`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:30:45.409585 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6d66ce7c-fec4-4082-af1b-d8c13e04096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:45.476331 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1433/2874] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:30:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:30:45.487787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746bff-c84e-4f94-88e9-07bbbd675a9b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:30:45.489533 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-d8746bff-c84e-4f94-88e9-07bbbd675a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:30:45.571706 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746bff-c84e-4f94-88e9-07bbbd675a9b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:30:45.575844 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc360681-462f-4857-9485-5aee4c4e3c86 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '107a90e384504e8d80264d6bf20f1d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:30:45.576514 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc360681-462f-4857-9485-5aee4c4e3c86 service glance] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:30:45.576804 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc360681-462f-4857-9485-5aee4c4e3c86 service glance] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:30:45.577109 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc360681-462f-4857-9485-5aee4c4e3c86 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:30:45.586393 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc360681-462f-4857-9485-5aee4c4e3c86 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:30:45.587875 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc360681-462f-4857-9485-5aee4c4e3c86 service glance] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:30:45.597071 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1442/2875] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:30:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:30:55.473129 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:30:55.592895 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=62,keystone:INSERT=2 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:31:03.274874 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66ce7c-fec4-4082-af1b-d8c13e04096b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:03.364775 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66ce7c-fec4-4082-af1b-d8c13e04096b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:03.373283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:03.374646 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:03.375003 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:03.375376 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] PATH_INFO: `/v3/users/d08d24e1e0164a68843862a56973c830` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:03.377345 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d08d24e1e0164a68843862a56973c830)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:03.379805 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:03.442289 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd08d24e1e0164a68843862a56973c830'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:03.442929 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd08d24e1e0164a68843862a56973c830'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:03.443545 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd08d24e1e0164a68843862a56973c830'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:03.452179 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd08d24e1e0164a68843862a56973c830'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:03.459821 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd08d24e1e0164a68843862a56973c830'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:03.478410 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1434/2876] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:31:03 2022] DELETE /identity/v3/users/d08d24e1e0164a68843862a56973c830 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:31:03.483281 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23141b4b-de25-4443-b4b8-1893ddbacfc3 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:03.483834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23141b4b-de25-4443-b4b8-1893ddbacfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:03.483834 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23141b4b-de25-4443-b4b8-1893ddbacfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:03.502284 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-23141b4b-de25-4443-b4b8-1893ddbacfc3 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:31:03.503511 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-23141b4b-de25-4443-b4b8-1893ddbacfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:03.563253 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1443/2877] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 04 09:31:03.573366 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:03.575169 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:03.635791 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56da5b-fbb5-48f1-987f-31d6589b4224 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:03.640104 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6753c-9326-4851-878b-9db6eeeb46fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:03.640751 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6753c-9326-4851-878b-9db6eeeb46fb service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:03.641032 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6753c-9326-4851-878b-9db6eeeb46fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:03.641302 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6753c-9326-4851-878b-9db6eeeb46fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:03.651156 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6753c-9326-4851-878b-9db6eeeb46fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:03.652594 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6753c-9326-4851-878b-9db6eeeb46fb service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:03.662269 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1435/2878] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:31:03.937652 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23141b4b-de25-4443-b4b8-1893ddbacfc3 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:03.939221 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-23141b4b-de25-4443-b4b8-1893ddbacfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:03.952283 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23141b4b-de25-4443-b4b8-1893ddbacfc3 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:03.957577 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:03.958498 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:03.958765 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:03.959035 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] PATH_INFO: `/v3/projects/f4be24e33fe0477b8fb0caf9541d8d1a` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:03.960386 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4be24e33fe0477b8fb0caf9541d8d1a)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:03.962306 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:04.012181 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4be24e33fe0477b8fb0caf9541d8d1a', 'request_id': 'req-21ace616-bb63-4941-ac85-fe4162260ea5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:04.023001 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4be24e33fe0477b8fb0caf9541d8d1a', 'request_id': 'req-21ace616-bb63-4941-ac85-fe4162260ea5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:04.029247 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4be24e33fe0477b8fb0caf9541d8d1a', 'request_id': 'req-21ace616-bb63-4941-ac85-fe4162260ea5'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:04.039942 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] The token cache is being invalidate because project f4be24e33fe0477b8fb0caf9541d8d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80283) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:31:04.040224 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:04.040887 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-21ace616-bb63-4941-ac85-fe4162260ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:04.042485 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1444/2879] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:31:03 2022] DELETE /identity/v3/projects/f4be24e33fe0477b8fb0caf9541d8d1a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:31:04.047561 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2b76e-aa7e-47f4-90ad-d95b32211482 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.047841 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2b76e-aa7e-47f4-90ad-d95b32211482 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.048120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2b76e-aa7e-47f4-90ad-d95b32211482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.067458 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c0d2b76e-aa7e-47f4-90ad-d95b32211482 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:31:04.069396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c0d2b76e-aa7e-47f4-90ad-d95b32211482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:04.088835 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:31:04.136327 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1436/2880] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:31:04.146067 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb5137-a1ab-4c2f-86e7-4543d1e23f22 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.146401 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb5137-a1ab-4c2f-86e7-4543d1e23f22 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.146684 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb5137-a1ab-4c2f-86e7-4543d1e23f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.165609 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-dbcb5137-a1ab-4c2f-86e7-4543d1e23f22 None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:31:04.166880 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-dbcb5137-a1ab-4c2f-86e7-4543d1e23f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:04.175413 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1445/2881] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:31:04.179630 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2b76e-aa7e-47f4-90ad-d95b32211482 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:04.189588 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2b76e-aa7e-47f4-90ad-d95b32211482 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:04.194817 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ceba2-48d0-40bf-8b49-b43cd5f40dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:04.195479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ceba2-48d0-40bf-8b49-b43cd5f40dde admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.196063 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ceba2-48d0-40bf-8b49-b43cd5f40dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.196354 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ceba2-48d0-40bf-8b49-b43cd5f40dde admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.197086 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ceba2-48d0-40bf-8b49-b43cd5f40dde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 04 09:31:04.197396 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ceba2-48d0-40bf-8b49-b43cd5f40dde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:04.199050 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ceba2-48d0-40bf-8b49-b43cd5f40dde admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:04.210831 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1437/2882] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 4 09:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 04 09:31:04.215474 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb5137-a1ab-4c2f-86e7-4543d1e23f22 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:04.227291 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb5137-a1ab-4c2f-86e7-4543d1e23f22 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:04.234612 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a57d9f-83d9-417e-b562-75ebefb32015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:04.235546 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a57d9f-83d9-417e-b562-75ebefb32015 admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.235907 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a57d9f-83d9-417e-b562-75ebefb32015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.236274 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a57d9f-83d9-417e-b562-75ebefb32015 admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.237372 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a57d9f-83d9-417e-b562-75ebefb32015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:04.241072 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a57d9f-83d9-417e-b562-75ebefb32015 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:04.276764 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1446/2883] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:31:04 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:31:04.281201 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ceba2-48d0-40bf-8b49-b43cd5f40dde admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:04.291824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ceba2-48d0-40bf-8b49-b43cd5f40dde admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:04.352710 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84956bd4-f08f-4734-b91b-ea516739f38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:04.353383 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84956bd4-f08f-4734-b91b-ea516739f38f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.353665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84956bd4-f08f-4734-b91b-ea516739f38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.353973 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84956bd4-f08f-4734-b91b-ea516739f38f admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.354665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84956bd4-f08f-4734-b91b-ea516739f38f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:04.356598 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84956bd4-f08f-4734-b91b-ea516739f38f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:04.379533 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1438/2884] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:31:04 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:31:04.383843 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a57d9f-83d9-417e-b562-75ebefb32015 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:04.395808 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a57d9f-83d9-417e-b562-75ebefb32015 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:04.401390 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f3e4e-58e1-4542-9b2a-c32445bdc248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:04.402096 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f3e4e-58e1-4542-9b2a-c32445bdc248 admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.402378 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f3e4e-58e1-4542-9b2a-c32445bdc248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.402646 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f3e4e-58e1-4542-9b2a-c32445bdc248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.403373 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f3e4e-58e1-4542-9b2a-c32445bdc248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:04.404978 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f3e4e-58e1-4542-9b2a-c32445bdc248 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:04.416048 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1447/2885] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 04 09:31:04.420243 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84956bd4-f08f-4734-b91b-ea516739f38f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:04.431535 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84956bd4-f08f-4734-b91b-ea516739f38f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:04.437116 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4377a18-518f-4308-897c-8f7e5d01a2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:04.437830 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4377a18-518f-4308-897c-8f7e5d01a2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.438244 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4377a18-518f-4308-897c-8f7e5d01a2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.438362 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4377a18-518f-4308-897c-8f7e5d01a2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.439133 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4377a18-518f-4308-897c-8f7e5d01a2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:04.440762 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4377a18-518f-4308-897c-8f7e5d01a2a2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:04.452083 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1439/2886] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:31:04.456192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f3e4e-58e1-4542-9b2a-c32445bdc248 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:04.466796 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f3e4e-58e1-4542-9b2a-c32445bdc248 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:04.472537 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf96f-3cc6-4dfc-b694-f62a7431f413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:04.473479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf96f-3cc6-4dfc-b694-f62a7431f413 admin admin] REQUEST_METHOD: `PUT` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.473739 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf96f-3cc6-4dfc-b694-f62a7431f413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.474060 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf96f-3cc6-4dfc-b694-f62a7431f413 admin admin] PATH_INFO: `/v3/projects/6355b48d589f4686b91799ba22989a45/users/1c24798e7a0f4f398c9b71e0ab028561/roles/0b98291295724747acd01dd0d2306e62` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.490314 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf96f-3cc6-4dfc-b694-f62a7431f413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6355b48d589f4686b91799ba22989a45, user_id=1c24798e7a0f4f398c9b71e0ab028561, role_id=0b98291295724747acd01dd0d2306e62)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:04.493016 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf96f-3cc6-4dfc-b694-f62a7431f413 admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:04.511822 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1448/2887] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:31:04 2022] PUT /identity/v3/projects/6355b48d589f4686b91799ba22989a45/users/1c24798e7a0f4f398c9b71e0ab028561/roles/0b98291295724747acd01dd0d2306e62 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:31:04.522827 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284394d-d535-4452-95fb-e17a56684f57 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.523123 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284394d-d535-4452-95fb-e17a56684f57 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.523402 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284394d-d535-4452-95fb-e17a56684f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.557186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3284394d-d535-4452-95fb-e17a56684f57 None None] MFA Rules not processed for user `1c24798e7a0f4f398c9b71e0ab028561`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:31:04.558359 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3284394d-d535-4452-95fb-e17a56684f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:04.632763 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1440/2888] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:31:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:31:04.637316 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf96f-3cc6-4dfc-b694-f62a7431f413 admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:04.660321 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:31:04.706965 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf96f-3cc6-4dfc-b694-f62a7431f413 admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:04.712214 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd98f6-b078-418e-9ec7-f54c29e07c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:04.712925 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd98f6-b078-418e-9ec7-f54c29e07c8a admin admin] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.713209 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd98f6-b078-418e-9ec7-f54c29e07c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.713486 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd98f6-b078-418e-9ec7-f54c29e07c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.714249 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd98f6-b078-418e-9ec7-f54c29e07c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:04.716236 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd98f6-b078-418e-9ec7-f54c29e07c8a admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:04.746915 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1449/2889] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 4 09:31:04 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:31:04.752797 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284394d-d535-4452-95fb-e17a56684f57 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:04.763989 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284394d-d535-4452-95fb-e17a56684f57 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:04.777192 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:31:04.823391 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe56aa-2d56-4625-aeba-33beeda0babf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:04.823976 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe56aa-2d56-4625-aeba-33beeda0babf admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.824256 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe56aa-2d56-4625-aeba-33beeda0babf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.824515 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe56aa-2d56-4625-aeba-33beeda0babf admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.825201 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe56aa-2d56-4625-aeba-33beeda0babf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:04.827173 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe56aa-2d56-4625-aeba-33beeda0babf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:04.848803 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1441/2890] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 4 09:31:04 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 04 09:31:04.853030 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd98f6-b078-418e-9ec7-f54c29e07c8a admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:04.863637 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd98f6-b078-418e-9ec7-f54c29e07c8a admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:04.868630 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ffad7-4223-42a1-8cad-31eb10953d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:04.869296 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ffad7-4223-42a1-8cad-31eb10953d0b admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.869573 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ffad7-4223-42a1-8cad-31eb10953d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.869849 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ffad7-4223-42a1-8cad-31eb10953d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.870623 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ffad7-4223-42a1-8cad-31eb10953d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:04.872402 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ffad7-4223-42a1-8cad-31eb10953d0b admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:04.884151 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1450/2891] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:31:04.888481 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe56aa-2d56-4625-aeba-33beeda0babf admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:04.899804 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe56aa-2d56-4625-aeba-33beeda0babf admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:04.905902 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a883a5-b3e8-4c61-bebb-c3c77554efc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:04.906701 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a883a5-b3e8-4c61-bebb-c3c77554efc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:04.907136 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a883a5-b3e8-4c61-bebb-c3c77554efc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:04.907419 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a883a5-b3e8-4c61-bebb-c3c77554efc7 admin admin] PATH_INFO: `/v3/projects/99c7e962dd7043ad9ad61d83cb7cfe20/users/acb8a35bebaa4d4aaac0d70f29efc354/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:04.924022 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a883a5-b3e8-4c61-bebb-c3c77554efc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c7e962dd7043ad9ad61d83cb7cfe20, user_id=acb8a35bebaa4d4aaac0d70f29efc354, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:04.927449 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a883a5-b3e8-4c61-bebb-c3c77554efc7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:04.943697 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1442/2892] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:31:04 2022] PUT /identity/v3/projects/99c7e962dd7043ad9ad61d83cb7cfe20/users/acb8a35bebaa4d4aaac0d70f29efc354/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:31:04.948076 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ffad7-4223-42a1-8cad-31eb10953d0b admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:04.959098 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ffad7-4223-42a1-8cad-31eb10953d0b admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:05.025729 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206b616-9e85-43c7-8ac0-7f9a7f92f83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:05.026434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206b616-9e85-43c7-8ac0-7f9a7f92f83f admin admin] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:05.026720 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206b616-9e85-43c7-8ac0-7f9a7f92f83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:05.026992 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206b616-9e85-43c7-8ac0-7f9a7f92f83f admin admin] PATH_INFO: `/v3/roles` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:05.027754 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206b616-9e85-43c7-8ac0-7f9a7f92f83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:05.029446 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206b616-9e85-43c7-8ac0-7f9a7f92f83f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:05.039643 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1451/2893] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 4 09:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 04 09:31:05.043881 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a883a5-b3e8-4c61-bebb-c3c77554efc7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:05.054952 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a883a5-b3e8-4c61-bebb-c3c77554efc7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:05.060694 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0f1c0-6683-4d36-9133-3e4ff9d794d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:05.061591 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0f1c0-6683-4d36-9133-3e4ff9d794d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:05.061877 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0f1c0-6683-4d36-9133-3e4ff9d794d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:05.062185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0f1c0-6683-4d36-9133-3e4ff9d794d8 admin admin] PATH_INFO: `/v3/projects/99c7e962dd7043ad9ad61d83cb7cfe20/users/acb8a35bebaa4d4aaac0d70f29efc354/roles/2302517080ac42578aa81c606dd0ff2f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:05.064339 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0f1c0-6683-4d36-9133-3e4ff9d794d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c7e962dd7043ad9ad61d83cb7cfe20, user_id=acb8a35bebaa4d4aaac0d70f29efc354, role_id=2302517080ac42578aa81c606dd0ff2f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:05.067479 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0f1c0-6683-4d36-9133-3e4ff9d794d8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:05.078513 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1443/2894] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 4 09:31:05 2022] PUT /identity/v3/projects/99c7e962dd7043ad9ad61d83cb7cfe20/users/acb8a35bebaa4d4aaac0d70f29efc354/roles/2302517080ac42578aa81c606dd0ff2f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:31:05.089184 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30c992-5b43-4a27-96fc-0844c46d1625 None None] REQUEST_METHOD: `POST` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:05.089509 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30c992-5b43-4a27-96fc-0844c46d1625 None None] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:05.089792 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30c992-5b43-4a27-96fc-0844c46d1625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:05.118201 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.auth.core [None req-2c30c992-5b43-4a27-96fc-0844c46d1625 None None] MFA Rules not processed for user `acb8a35bebaa4d4aaac0d70f29efc354`. Rule list: `[]` (Enabled: `True`). {{(pid=80283) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:31:05.119416 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-2c30c992-5b43-4a27-96fc-0844c46d1625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:05.194884 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1452/2895] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:31:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:31:05.205761 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0f1c0-6683-4d36-9133-3e4ff9d794d8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:05.207727 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f2d0f1c0-6683-4d36-9133-3e4ff9d794d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:05.267081 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0f1c0-6683-4d36-9133-3e4ff9d794d8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:05.271185 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f50637-0ac0-428a-83d4-0b95412f950c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f34c35a9ba4376a9b55ec5e23cf35f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:05.271824 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f50637-0ac0-428a-83d4-0b95412f950c service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:05.272117 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f50637-0ac0-428a-83d4-0b95412f950c service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:05.272388 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f50637-0ac0-428a-83d4-0b95412f950c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:05.332886 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f50637-0ac0-428a-83d4-0b95412f950c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:05.334283 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f50637-0ac0-428a-83d4-0b95412f950c service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:05.343790 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1444/2896] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:31:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:31:07.903169 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30c992-5b43-4a27-96fc-0844c46d1625 None None] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:07.922768 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:31:07.984419 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30c992-5b43-4a27-96fc-0844c46d1625 None None] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:07.989641 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:07.990878 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:07.991167 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:07.991444 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] PATH_INFO: `/v3/users/1c24798e7a0f4f398c9b71e0ab028561` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:07.992715 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c24798e7a0f4f398c9b71e0ab028561)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:07.994692 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:08.045814 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c24798e7a0f4f398c9b71e0ab028561'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.046892 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c24798e7a0f4f398c9b71e0ab028561'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.056711 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c24798e7a0f4f398c9b71e0ab028561'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.064190 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c24798e7a0f4f398c9b71e0ab028561'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.064846 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c24798e7a0f4f398c9b71e0ab028561'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.085856 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1453/2897] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:31:07 2022] DELETE /identity/v3/users/1c24798e7a0f4f398c9b71e0ab028561 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:31:08.092412 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d81ea-06df-4b28-8ee4-191503ad37bc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:08.092948 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d81ea-06df-4b28-8ee4-191503ad37bc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:08.093055 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d81ea-06df-4b28-8ee4-191503ad37bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:08.112305 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-426d81ea-06df-4b28-8ee4-191503ad37bc None None] MFA Rules not processed for user `78c2f3799fc8459a9a6e05f14a552498`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 04 09:31:08.113527 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-426d81ea-06df-4b28-8ee4-191503ad37bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:08.135719 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:31:08.179178 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1445/2898] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 4 09:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 04 09:31:08.191515 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:08.193158 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:08.251439 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca450ec1-6b51-49a0-aa1d-257ee1dad99f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:08.255192 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d371308-1ed6-40fc-931e-70a8cbb12dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98649aa38f764af0b41ff5594b94a989', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:08.255920 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371308-1ed6-40fc-931e-70a8cbb12dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:08.256204 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371308-1ed6-40fc-931e-70a8cbb12dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:08.256479 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d371308-1ed6-40fc-931e-70a8cbb12dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:08.266434 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d371308-1ed6-40fc-931e-70a8cbb12dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:08.267864 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d371308-1ed6-40fc-931e-70a8cbb12dc7 service neutron] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:08.280787 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1454/2899] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:31:08.567560 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d81ea-06df-4b28-8ee4-191503ad37bc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:08.569665 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-426d81ea-06df-4b28-8ee4-191503ad37bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:08.589098 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d81ea-06df-4b28-8ee4-191503ad37bc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:08.594702 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:08.595662 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:08.595849 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:08.596140 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] PATH_INFO: `/v3/projects/6355b48d589f4686b91799ba22989a45` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:08.597349 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6355b48d589f4686b91799ba22989a45)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:08.599294 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:08.651908 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6355b48d589f4686b91799ba22989a45', 'request_id': 'req-d7875d7e-a6d1-4254-8eac-844f1bc7144d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.662120 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6355b48d589f4686b91799ba22989a45', 'request_id': 'req-d7875d7e-a6d1-4254-8eac-844f1bc7144d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.669166 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6355b48d589f4686b91799ba22989a45', 'request_id': 'req-d7875d7e-a6d1-4254-8eac-844f1bc7144d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.681291 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] The token cache is being invalidate because project 6355b48d589f4686b91799ba22989a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:31:08.681473 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.682180 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.683814 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1446/2900] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:31:08 2022] DELETE /identity/v3/projects/6355b48d589f4686b91799ba22989a45 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:31:08.688146 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d371308-1ed6-40fc-931e-70a8cbb12dc7 service neutron] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:08.698602 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d371308-1ed6-40fc-931e-70a8cbb12dc7 service neutron] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:08.736031 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:31:08.771527 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:08.772735 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:08.772972 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:08.773304 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] PATH_INFO: `/v3/users/acb8a35bebaa4d4aaac0d70f29efc354` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:08.775394 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] RBAC: Authorizing `identity:delete_user(user_id=acb8a35bebaa4d4aaac0d70f29efc354)` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:08.779153 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:08.834607 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acb8a35bebaa4d4aaac0d70f29efc354'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.835425 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acb8a35bebaa4d4aaac0d70f29efc354'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.843882 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acb8a35bebaa4d4aaac0d70f29efc354'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.851709 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acb8a35bebaa4d4aaac0d70f29efc354'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.852359 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.notifications [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acb8a35bebaa4d4aaac0d70f29efc354'} {{(pid=80283) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:08.869322 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1455/2901] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 4 09:31:08 2022] DELETE /identity/v3/users/acb8a35bebaa4d4aaac0d70f29efc354 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 04 09:31:09.173637 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:31:09.175373 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:31:09.189286 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7875d7e-a6d1-4254-8eac-844f1bc7144d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:31:09.254186 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78c2f3799fc8459a9a6e05f14a552498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f0a53c420a84618ab851430c484608a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:31:09.255329 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:31:09.255738 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:31:09.256141 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] PATH_INFO: `/v3/projects/99c7e962dd7043ad9ad61d83cb7cfe20` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:31:09.257919 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c7e962dd7043ad9ad61d83cb7cfe20)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:31:09.260966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:31:09.315668 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c7e962dd7043ad9ad61d83cb7cfe20', 'request_id': 'req-2e841683-07d9-41c7-aeeb-419c8bf23fc1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:09.317311 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:31:09.325985 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c7e962dd7043ad9ad61d83cb7cfe20', 'request_id': 'req-2e841683-07d9-41c7-aeeb-419c8bf23fc1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:09.334119 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c7e962dd7043ad9ad61d83cb7cfe20', 'request_id': 'req-2e841683-07d9-41c7-aeeb-419c8bf23fc1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:09.344966 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] The token cache is being invalidate because project 99c7e962dd7043ad9ad61d83cb7cfe20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 04 09:31:09.345260 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:09.346020 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2e841683-07d9-41c7-aeeb-419c8bf23fc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 04 09:31:09.347667 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1447/2902] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 4 09:31:09 2022] DELETE /identity/v3/projects/99c7e962dd7043ad9ad61d83cb7cfe20 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 04 09:31:18.865963 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:31:19.341639 dv-cinder-819149-7-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 04 09:33:07.037024 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] Authenticating user token {{(pid=80283) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 04 09:33:07.038698 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:33:07.122788 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321923a-f783-472a-850b-037f8be3e98f admin admin] Validating token access rules against request {{(pid=80283) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 04 09:33:07.195517 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c07f6-6f9f-491f-8d91-127430c85ef3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff2305249a242639e1415094bf9df33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c526301d4c7441396af366e54d9b3b5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80283) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 04 09:33:07.196189 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c07f6-6f9f-491f-8d91-127430c85ef3 service placement] REQUEST_METHOD: `GET` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 04 09:33:07.196472 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c07f6-6f9f-491f-8d91-127430c85ef3 service placement] SCRIPT_NAME: `/identity` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 04 09:33:07.196801 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c07f6-6f9f-491f-8d91-127430c85ef3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80283) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 04 09:33:07.199140 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.fernet_utils [None req-698c07f6-6f9f-491f-8d91-127430c85ef3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80283) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 04 09:33:07.211773 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c07f6-6f9f-491f-8d91-127430c85ef3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 04 09:33:07.213681 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c07f6-6f9f-491f-8d91-127430c85ef3 service placement] RBAC: Authorization granted {{(pid=80283) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 04 09:33:07.225248 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: [pid: 80283|app: 0|req: 1456/2903] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 4 09:33:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 04 09:33:17.220477 dv-cinder-819149-7-iscsi devstack@keystone.service[80283]: DEBUG dbcounter [-] [80283] Writing DB stats keystone:SELECT=58 {{(pid=80283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}